Authoring system and authoring method, and storage medium

ABSTRACT

An action editing window ( 200 ) is formed as a time table of a two-dimensional time line style, made up by a horizontal time axis and a vertical channel. Within the time line table, respective constituent elements of an action of a robot, such as a time ruler ( 201 ), a keyframe channel ( 202 ), a motion channel ( 203 ), a sound channel ( 204 ) or an LED operation channel ( 205 ), are chronologically displayed. A user is able to edit as synchronization between the respective components of the action is checked. This supports the creation and the editing of a series of commands and data stating a pattern of robot movements.

TECHNICAL FIELD

[0001] This invention relates to an authoring system and an authoringmethod for supporting creation and editing of data in accordance with apreset scenario and, more particularly, to an authoring system and anauthoring method for supporting creation and editing of a sequence ofcommands/data stating a preset operating pattern of a robot.

[0002] More particularly, it relates to an authoring system and anauthoring method for supporting creation and editing of an operatingpattern using a set of components prescribing the operating states of arobot, and to an authoring system and an authoring method for arrayingthe respective components on a computer display for supporting creationand editing of the operating pattern.

BACKGROUND ART

[0003] A mechanical apparatus for performing movements simulating themovement of the human being, using electrical or magnetic operation, istermed a “robot”. The etymology of the term robot is said to be “ROBOTA”(slave machine) of the Slavic language. The robots started to be usedwidely in this country towards the end of the sixtieth. Most of therobots used were industrial robots, such as manipulators or transportingrobots, aimed at automation or unmanned operations in plants.

[0004] In recent years, researches and development in legged mobilerobots, including pet type robots, simulating the bodily mechanism ormovements of animals, such as quadruples, e.g., dogs or cats, orso-called humanoid robots, simulating the bodily mechanism or movementsof animals erected and walking on feet, such as human being, areprogressing, and expectations are made of practical utilization of thesetypes of robots. The legged mobile robots, while being unstable anddifficult to control as to their orientation or walking, as compared tocrawler or tire type robots, are superior in their performance inclimbing up and down a ladder or a staircase, in riding over obstaclesor walking or in running flexibly on a leveled or non-leveled terrain.

[0005] The standstill type robots, installed and used at a fixed place,such as armed robots, are in operation only in a stationary or localworking space, such as for assembling or sorting of component parts. Onthe other hand, the mobile robots are not limited as to working spaceand are movable on a preset or undefined path in an unrestricted fashionto perform operations to take the place of human operators or to offervariegated services to take the place of the human being, dogs or otherliving organisms.

[0006] One of the usages of the legged mobile robot is substitution of avariety of operations dangerous or difficult to perform in industrial orproductive activities. Examples of these usages include substitution ofa variety of operations dangerous or difficult to perform, such asmaintenance operations in nuclear power plants, thermal power plants orpetrochemical plants, transporting or assembling operations inmanufacturing plants, cleaning in high-rise buildings or rescuing e.g.,on the site of conflagrations.

[0007] Another usage of the legged mobile robot is life-related usage,such as co-living with or rendering entertainment for the human being,rather than acting as the operational support described above. This sortof robot emulates a rich feeling expression exploiting the four limbs oran operation mechanism of the legged walking animal of high intellect,such as human being or dog (pet). Moreover, this sort of the robot isrequested not only to execute the pre-set operating pattern measlyfaithfully, but also to realize vivid expressions dynamically respondingto the language or behavior of a counterpart operator, such as‘praising’, ‘scolding’ or ‘patting’.

[0008] With a conventional toy machine, the relationship between theuser operation and the responsive operation is fixed, such that thetoy's movements cannot be changed to suit to the user's liking. Theresult is that the user gets tired of the toy, simply repeating only thesame sort of the operations, sooner or later.

[0009] Conversely, an intellectual robot owns a behavioral or learningmodel, emanating from movements, and determines its movements as itvaries the model based on the input information from outside, such asspeech, image or tactile feeling, to realize autonomous thinking oroperation control. The feeling or the autonomous behavior of the robotitself can be represented by providing the robot with the feeling orinstinct model. By the robot owning a picture inputting device or with aspeech input/output device and executing the processing of picture orspeech recognition, it is possible to realize realistic communicationbetween the robot and the human being on a higher intellectual level.

[0010] In addition, by affording the possibility of changing the modelresponsive to the detection of a stimulus from outside, such as user'sactuation, that is the ‘learning effect’, it is possible to present tothe user a movement pattern which is not tiresome to the user or whichadapts itself more satisfactorily to the user's liking.

[0011] Nowadays, the legged mobile robot owns a high informationprocessing capability, such that the robot itself may be thought of as asort of a computing system. Consequently, the sequence of the highlyadvanced sophisticated operations, constructed by the operating pattern,realized on the robot, or by the combination of plural basic operatingpatterns, may be constructed by an operation similar to that performedin the computer programming.

[0012] In near future, it may be anticipated that the robot becomes morepopular to become more diffused not only in industrial circles but alsoin households and in our everyday life. In particular, as concernsproducts which pursue entertainment aspects, it may be anticipated thatconsumers at large, not having profound knowledge in computer orcomputer programming, purchase and use the robot. It may be thought tobe desirable to provide the users at large with a tool which helpscreate and edit the operating sequence of the robot relatively readilyand efficiently by interactive processing, that is with a so-calledauthoring system.

DISCLOSURE OF THE INVENTION

[0013] It is therefore an object of the present invention to provide anexcellent authoring system and method which are able to help create andedit a sequence of command/data stating a preset operating pattern ofthe robot.

[0014] It is another object of the present invention to provide anexcellent authoring system and method which are able to help create andedit an operating pattern using a set of components prescribing theoperating states of a robot.

[0015] It is a further object of the present invention to provide anexcellent authoring system and method which allow to array variouscomponents on a computer display to help create and edit an operatingpattern for the robot.

[0016] An authoring system according to the present invention is such asystem supporting the creation and editing of an action of a multi-jointstructure comprised of a combination of a plurality of chronologicaldata, including a user inputting unit for inputting commands or datafrom a user, a user presenting unit for presenting an editing area inwhich chronological data making up the action are arrayedchronologically along the time axis, and a chronological data editingunit provided for each chronological data making up the action. Thechronological data editing unit creates or edits relevant chronologicaldata based on a user input through the user presenting unit.

[0017] The ‘system’ herein means a logical assembly of plural devices orfunctional modules realizing specified functions, without regard towhether or not the individual devices or functional modules are enclosedin one casing.

[0018] The multi-joint structure is a legged robots having two legs,four legs and so forth and which are constructed from plural jointactuators. Alternatively, the multi-joint structure may be other typesof multi-joint robots, or a character for producing an animation, basedon joint movements, as a result of computer graphics.

[0019] The chronological data making up the action may be motion datastating the chronological movements of respective joints of themulti-joint structure. The motion data may prescribe chronologicalmovements of respective joints of the multi-joint structure and, byarraying two or more keyframes, each representing the multi-jointstructure striking a preset pose on the time axis, may smoothly join therespective keyframes.

[0020] Another chronological data making up the action is sound dataoutput as sound in timed relation to reproduction of the action. Thesound data may be stated in the MIDI (Musical Instrumental DigitalInterface) form or WAVE form.

[0021] Still another example of the chronological data making up theaction is indicator display data stating the turning on/off of displayindicators displayed in timed relation to action reproduction. Theindicator display data may be stated in the MIDI (Musical InstrumentalDigital Interface) form.

[0022] The editing area presented by the user presenting unit may becomprised of chronological-data-based chronological data displaychannels. These data extend along the horizontally arrayed time axis,with the display channels being arrayed in the vertical direction.

[0023] That is, with the authoring system according to the presentinvention, the chronological data making up an action of the mobilerobot may be displayed as they are arrayed along the time axis on thetime table of the two-dimensional time line style. Consequently, thecreation and editing may be prosecuted as synchronization between therespective chronological data is visually checked so that a workingenvironment efficient and intuitively comprehensible may be provided foraction editing.

[0024] The editing area presented by the user presenting unit mayinclude one or more time display lines extending in the verticaldirection for specifying the relevant time as prescribed by the timeruler. In such case, the state of progress between the respective timeindicating lines or the state of synchronization between the respectivechronological data channels may be visually checked with the timeindicating lines as a reference.

[0025] The editing area presented by the user presenting unit mayinclude a vertically extending current time display line for indicatingthe current time on the time axis prescribed by the time ruler. Thecurrent time display line may be configured for being shifted on thetime ruler to a position corresponding to user actuation.

[0026] The editing area presented by the user presenting unit mayinclude a display window for previewing movements of the multi-jointstructure at the current time.

[0027] The editing area presented by the user presenting unit mayinclude a keyframe channel for indicating the respective keyframes orthumbnails thereof along the time axis as prescribed by the time ruler.In the keyframe channel, the time changes of the keyframe may beaccommodated so as to follow up with keyframe or thumbnail dragging. Apose editing picture for editing a relevant pose may be startedresponsive to the operation of selection by the user of the keyframe orthumbnail within the keyframe channel.

[0028] The editing area presented by the user presenting unit may alsoinclude a motion channel for editing and displaying the contents of amotion along the time axis prescribed by the time ruler. This motionchannel is constructed by arraying respective timing charts,representing the chronological movements of respective joints of themulti-joint structure in the vertical direction. It is also possible toaccommodate changes in the joint movements at a relevant time point soas to follow up with dragging on the timing chart in the motion channel.

[0029] The editing area presented by the user presenting unit may alsobe provided with a sound channel for indicating the contents of thesound along the time axis as prescribed by the time ruler, or with adisplay indicator channel for displaying the contents of the indicatordisplay data along the time axis prescribed by the time ruler.

[0030] The user presenting unit may also be provided with a soundediting area for displaying and editing the sound contents along thetime axis prescribed by the time ruler.

[0031] The sound editing area may include a score channel constituted bya basic grid along the time axis direction and by a piano keyboard. Inthis case, a score is formed by the sound duration as the reference ofthe time axis and the pitch of the piano keys. On the score channel, thesound may be edited by arraying the color corresponding to sound notesin a cell corresponding to the desired time point and the sound scale.The velocity channel may also be demonstrated for representing theintensity of each sound along the time axis.

[0032] The user presenting unit may further include an indicator editingarea for displaying and editing the contents of indicator data along thetime axis prescribed by the time ruler.

[0033] The indicator editing area includes a score channel constitutedby a list of sites of arraying of display indicators and by the basicgrid along the time axis direction. In such case, a score may be editedfrom one site list to another by indicating the turned-on state of theindicator in each site on the time axis on a score channel.

[0034] The user presenting unit may further display a preview window forvisually checking an action of the multi-joint structure generated basedon each chronological data as edited by the chronological data editingunit.

[0035] The preview window may include a 3D view area forthree-dimensional display of movements of the multi-joint structuregenerated based on motion data. The preview window may include a displayindicator operation preview area for displaying the operation of theindicators corresponding to the indicator display data in synchronismwith the preview of other chronological data.

[0036] The user presenting unit may further display a pose window forediting the pose of the multi-joint structure by the GUI operation.

[0037] The pose window may include an entity specifying area fordisplaying the multi-joint structure in a developed plan view foraccepting user selection of editable sites. The authoring pose windowmay include a list specifying area for displaying a list of editablesites of the multi-joint structure and setting values thereof. The posewindow may also include a setting value area for displaying a list ofthe names of setting sites, setting values, settable maximum values andsettable minimum values of editable sites of the multi-joint structure.The pose window may also include a 3D display area for 3D displaying afull image of the multi-joint structure generated by 3D graphics and foraccepting user selection on the 3D display.

[0038] The authoring system may also include data inputting means forinputting chronological data making up an action of the multi-jointstructure from outside. The pose window may also display the posegenerated based on data input from the data inputting means.

[0039] The user presenting unit may further display a motion previewwindow arraying one or more key frames making up the motion orthumbnails thereof in accordance with the chronological sequence used inmotion reproduction.

[0040] An authoring method according to the present invention is such amethod for supporting the creation and editing of an action of amulti-joint structure comprised of a combination of a plurality ofchronological data, and includes a user presenting step of presenting anediting area in which the chronological data making up the action arearrayed chronologically along the time axis and a chronological dataediting step for creating or editing relevant chronological data basedon a user input through the editing area, provided for eachchronological data making up the action, via the user presenting step.

[0041] One of the chronological data making up the action is motion datastating the chronological movements of respective joints of the multijoint structure. This motion data may prescribe chronological movementsof respective joints of the multi-joint structure. The motion datasmoothly joins two or more keyframes by arraying these keyframes eachrepresenting the multi-joint structure striking a preset pose on thetime axis.

[0042] Another example of the chronological data making up the action issound data output as sound in timed relation to reproduction of theaction. The sound data may be stated in the MIDI (Musical InstrumentalDigital Interface) form or WAVE form. Still another example ofchronological data making up the action is display indicator displaydata stating the turning on/off of display indicators, with the displaydata being displayed in timed relation to reproduction of the action.The indicator display data may be stated in the MIDI (MusicalInstrumental Digital Interface) form.

[0043] The editing area presented in the user presenting step may becomprised of chronological-data-based chronological data displaychannels, with the data extending along the horizontally arrayed timeaxis, the channels being arrayed in the vertical direction.

[0044] That is, with the authoring method according to the presentinvention, the chronological data forming the action of the mobilerobot, such as motion data, sound data or LED operating data, may bearrayed and displayed along the time axis on the timetable of thetwo-dimensional time line style to render it possible to visually checkthe synchronization between respective chronological data in prosecutingthe creation or editing. The result is provision of a workingenvironment for action edition which is more efficient andcomprehensible intuitively.

[0045] The editing area presented in the user presenting step mayinclude a time ruler comprised of graduations used to display the timeaxis in real-time. In such case, it is possible to readily visuallyrecognize the synchronization between the respective chronological datachannels and the time axis.

[0046] The editing area presented by the user presenting step mayinclude one or more time display lines extending in the verticaldirection for indicating the relevant time prescribed by a time ruler.In such case, the state of progress between the respective timeindicating lines or the state of synchronization between the respectivechronological data channels may be visually checked with the timeindicating lines as a reference.

[0047] The user presenting step may present an editing area having avertically extending current time display line for indicating thecurrent time on the time axis prescribed by the time ruler. There mayfurther be included a further step of shifting the current time displayline on the time ruler to a position corresponding to user actuation.

[0048] There may further be provided a step of presenting a displaywindow for previewing movements of the multi-joint structure at thecurrent time.

[0049] The user presenting step may further present an editing areahaving a keyframe channel for demonstrating each keyframe or itsthumbnail in accordance with the time axis prescribed by the time ruler.In such case, there may further be provided a step of changing thekeyframe time so as to follow up with dragging of the keyframe or itsthumbnail in the keyframe channel. There may also be provided a step ofstarting a pose editing picture for editing the relevant pose responsiveto the user's operation for selecting the keyframe or its thumbnail inthe keyframe channel.

[0050] The user presenting step may also present an editing areaprovided with a motion channel for editing and displaying the motioncontents along the time axis prescribed by the time ruler. In such case,the respective timing charts representing the chronological movements ofthe respective joints of the multi-joint structure may be arrayedvertically to represent the motion channel. There may further beprovided a step of changing the joint movements at the relevant timepoints such as to follow up with dragging on the timing charts in themotion channel.

[0051] In the user presenting step, there may be presented an editingarea having a sound channel for demonstrating the sound contents alongthe time axis prescribed by the time ruler. Alternatively, the userpresenting step may present an editing area having a display indicatorchannel for displaying the contents of the indicator display data alongthe time axis prescribed by the time ruler.

[0052] The authoring method may further include a step of displaying thesound editing area for displaying and editing the sound contents alongthe time axis prescribed by the time ruler.

[0053] The sound editing area may include a score channel constituted bya basic grid along the time axis direction and a piano keyboard. In suchcase, a score is formed by the sound duration as the reference of thetime axis and by the pitch of the piano keys. On the score channel, thesound may be edited by arraying the color corresponding to sound notesin a cell corresponding to the desired time point and the sound scale onthe score channel. The sound editing area may also include a velocitychannel for representing the intensity of each sound along the timeaxis.

[0054] The authoring method may further include a step of demonstratinga display indicator editing area for displaying and editing the contentsof the indicator display area along the time axis prescribed by the tieruler.

[0055] The indicator editing area may include a score channelconstituted by a list of arraying sites of the display indicators and bythe basic grid along the time axis direction. In such case, the scoremay be edited from one site list to another by demonstrating theturned-on state of the display indicators in respective sites on thetime axis on a score channel.

[0056] The authoring method may further include a step of displaying apreview window for visually checking an action of the multi-jointstructure generated based on each chronological data as edited in thechronological data editing step.

[0057] The preview window indicating step may three-dimensionallydisplay movements of the multi-joint structure, generated based onmotion data, in a 3D view area. Alternatively, the preview windowindicating step may display the operation of the display indicatorscorresponding to the indicator display data in synchronism with thepreview of other chronological data.

[0058] The authoring method may further include a step of displaying apose window for editing the pose of the multi-joint structure by the GUIoperation.

[0059] The pose window may include an entity specifying area fordisplaying the multi-joint structure in a developed plan view foraccepting user selection of editable sites, a list specifying area fordisplaying a list of editable sites of the multi-joint structure andsetting values thereof, a setting value area for displaying a list ofthe names of setting sites, setting values, settable maximum values andsettable minimum values of editable sites of the multi-joint structure,or a 3D display area for 3D demonstrating a full image of themulti-joint structure generated by 3D graphics and for accepting userselection of the editable sites on the 3D display.

[0060] The authoring method may further include a data inputting step ofinputting chronological data making up an action of the multi-jointstructure from outside. The pose window displays the pose generatedbased on data input from the data inputting step.

[0061] The authoring method may also include a step of displaying amotion preview window arraying one or more key frames making up themotion or thumbnails thereof in accordance with the chronologicalsequence used in motion reproduction.

[0062] A recording medium according to the present invention hasphysically stored thereon, in a computer-readable form, a computersoftware stating the processing for supporting the creation and editingof an action of a multi-joint structure, comprised of the combination ofplural chronological data, on a computer system. The computer softwareincludes a user presenting step of presenting an editing area in whichchronological data making up the action are arrayed chronologicallyalong the time axis, and a chronological data editing step of creatingor editing relevant chronological data based on a user input through anediting area by the user presenting step. The editing area is providedfor each chronological data making up the action.

[0063] The recording medium according to the present invention is amedium for physically furnishing a computer software in a computerreadable form to the general-purpose computer system capable ofexecuting a variety of program codes. Such medium is a removableportable recording medium, exemplified by a CD (Compact Disc), an FD(Floppy Disc), a MD (Mini-Disc) or a MO (Magneto-Optical Disc). It isalso possible to furnish the computer software to a specified computersystem in a computer readable form through a transmission medium, suchas a network, which may be a wireless or cable network.

[0064] This recording medium represents a definition of the cooperativerelationship between the computer software an the recording medium instructure or function in order to achieve the functions of the presetcomputer software on the computer system. Stated differently, byinstalling a preset compute software on a computer system through arecording medium according to the third aspect of the present invention,the cooperative action may be manifested on the computer system toachieve the operation and effect similar to those of the authoringsystem and method in the first and second aspects of the presentinvention.

[0065] Other objects, features and advantages of the present inventionwill become more apparent from reading the embodiments of the presentinvention as shown in the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0066]FIG. 1 is a perspective view of a mobile robot 1 performingwalking on four legs, embodying the present invention.

[0067]FIG. 2 schematically shows the structure of an electrical andcontrolling system of the mobile robot 1.

[0068]FIG. 3 is a detailed view of a controller 20.

[0069]FIG. 4 schematically shows a hardware structure of a host computer100.

[0070]FIG. 5 schematically shows the hardware structure of the hostcomputer 100.

[0071]FIG. 6 shows a project window.

[0072]FIG. 7 indicates a submenu item of a menu ‘file’ in the projectwindow.

[0073]FIG. 8 indicates a submenu item of a menu ‘material’ in theproject window.

[0074]FIG. 9 shows an action editing window.

[0075]FIG. 10 shows a submenu ‘file’ in an action editing window.

[0076]FIG. 11 shows a submenu item of a menu ‘editing’ in the actionediting window.

[0077]FIG. 12 shows a submenu item of a menu ‘material’ in the actionediting window.

[0078]FIG. 13 shows the action editing window with a key frame channelopen (key frame detail channel).

[0079]FIG. 14 shows the action editing window with a motion channel open(motion detail channel).

[0080]FIG. 15 shows a sound detail window for editing the sound file ofthe MIDI form.

[0081]FIG. 16 shows a sum-menu item of the menu ‘file’ in the sounddetail window for MIDI form.

[0082]FIG. 17 shows a sum-menu item of the menu ‘edit’ in the sounddetail window for MIDI form.

[0083]FIG. 18 shows a sum-menu item of the menu ‘set’ in the sounddetail window for MIDI form.

[0084]FIG. 19 schematically shows the structure of the sound detailwindow for demonstrating a sound file of the WAVE form.

[0085]FIG. 20 shows a submenu item of the menu ‘file’ in the sounddetail window for WAVE form.

[0086]FIG. 21 shows a submenu item of the menu ‘edit’ in the sounddetail window for WAVE form.

[0087]FIG. 22 schematically shows the structure of an LED detail windowfor demonstrating an LED operation file.

[0088]FIG. 23 shows a submenu item of the menu ‘file’ in the LEDoperation file.

[0089]FIG. 24 shows a submenu item of menu ‘file’ in the LED operationfile.

[0090]FIG. 25 schematically shows the structure of a preview window formonitoring an action edited in the action editing window.

[0091]FIG. 26 schematically shows the structure of a pose window forediting a 3D pose of a mobile robot by a GUI operation.

[0092]FIG. 27 is a flowchart for pre-view on the pose window of thecontents directly taught on an actual machine.

[0093]FIG. 28 schematically shows the structure of motion previewer forpreviewing the motion.

[0094]FIG. 29 is a block diagram of the authoring system.

BEST MODE FOR CARRYING OUT THE INVENTION

[0095] Referring to the drawings, certain preferred embodiments of thepresent invention will be explained in detail.

[0096] A. Robot Structure

[0097]FIG. 1 shows the appearance and the structure of the mobile robot1 performing legged movements on four legs, embodying the presentinvention. As shown, the robot 1 is a multi-articulated mobile robot,constructed after the shape and the structure of a four-legged animal asa model. In particular, the mobile robot 1 of the present embodiment hasthe aspect of a pet type robot, designed after the shape and thestructure of a dog, as representative of a pet animal, and is able toco-exist with the human being in the human living environment and toexpress the operations responsive to the user's operations.

[0098] The mobile robot 1 has a trunk unit 2, a head unit 3, a tail 4and four limbs, that is leg units 6A to 6D.

[0099] The head unit 3 is arranged at approximately the fore upper endof the trunk unit 2 through a neck joint 7 having the degrees of freedomin the roll, pitch and yaw directions, as shown. On the head unit 3,there are loaded a CCD (charge coupled device) camera 15, equivalent tothe ‘eye’ of the dog, a microphone 16, equivalent to its ‘ear’, aloudspeaker 17, equivalent to its ‘mouth’, a touch sensor 18, equivalentto its tactile sense, and a plural number of LED indicators 19. Theremay also be loaded other sensors making up the five senses of the livingbeing.

[0100] The tail 4 is mounted for bending or pivoting movements on theapproximately rear upper end of the trunk unit 2 through a tail joint 8having the degrees of freedom of roll and pitch axes.

[0101] A pair of leg units 6A, 6B make up fore legs, whilst leg units6C, 6D make up hind legs. The leg units 6A to 6D are each comprised ofcombination of thigh units 9A to 9D and shank units 10A to 10D, and aremounted at fore and aft and left and right corners of the bottom surfaceof the trunk unit 2. The thigh units 9A to 9D are connected to presetsites of the trunk unit 2 by hip joints 11A to 11D having the degrees offreedom about roll, pitch and yaw axes. The thigh units 9A to 9D and theshank units 10A to 10D are interconnected by knee joints 12A to 12Dhaving the degrees of freedom about the roll and pitch axes.

[0102] With the above-described legged mobile robot 1, constructed asdescribed above, the respective joint actuators may be driven under thecommands from a controller, as later explained, for swinging the headunit 3 in the up-and-down or in the left-and-right directions, swingingthe tail 4 or driving the leg units 6A to 6D in concert synchronouslyfor realizing such movements as running or walking.

[0103] Meanwhile, the degree of freedom of the joints of the mobilerobot 1 is furnished by rotational driving of joint actuators, notshown, arranged on the respective axes. It should be noted that thenumber of the degree of freedom of the joints of the mobile robot 1 isoptional and may be freely determined without departing from the scopeof the present invention.

[0104]FIG. 2 schematically shows the structure of the electrical andcontrolling system of the mobile robot 1. As shown, the mobile robot 1is made up of a controller 20, an input/output unit 40, a driving unit50 and a power source unit 60, performing the comprehensive control ofthe overall operations and the processing of other data. Theserespective components are hereinafter explained.

[0105] The input/output unit 40 includes a CCD camera 15, equivalent tothe ‘eye’ of the mobile robot 1, a microphone 16, equivalent to its‘ear’, a touch sensor 18, equivalent to its tactile sense, and a pluralnumber of sensors equivalent to the five senses. As the output unit,there are also provided a loudspeaker 17 equivalent to the mouth, and anumber of LED indicators 19 forming facial expressions based on thecombinations of lighting/extinguishment or the lighting timing. Theseoutput units are able to express the feedback to a user from the mobilerobot 1 in a form other than mechanical movement patterns employinge.g., the legs.

[0106] The mobile robot 1, provided with the camera 15, is able torecognize the shape or the color of an optional object present in aworkspace. The mobile robot 1 may also be provided with a receipt devicefor receiving transmitted waves, such a IR rays, sound waves, ultrasonicwaves or electrical waves, in addition to visual means, such as cameras.In this case, the positions or orientation from the transmitting sourcemay be measured based on an output of a sensor detecting the transmittedwaves.

[0107] The driving unit 50 is a functional block for realizingmechanical movements of the mobile robot 1 in accordance with a presetmovement pattern as commanded by the controller 20. The driving unit iscomprised of driving units provided on each of the roll, pitch and yawaxes of the neck joint 7, tail joint 8, hip joints 11A to 11D and theknee joints 12A to 12D. In the illustrated instance, the mobile robot 1has n degrees of freedom, at the respective joints, so that the drivingunit 50 is made up of n driving units. Each driving unit is comprised ofcombinations of a motor 51 for producing rotational movements aboutpreset axes, an encoder 52 for detecting rotational positions of themotor 51 and a driver 53 for adaptively controlling the rotationalpositions and rotational speeds of the motor 51 based on the outputs ofthe encoder 52.

[0108] The power source unit 60, as its name implies, is a functionalmodule responsible for current supply to respective electrical circuitswithin the mobile robot 1. The mobile robot 1 of the instant embodimentis of an autonomous driving system, employing a battery. The powersource unit 60 includes a battery for charging 61 and acharging/discharging controller 62 for supervising thecharging/discharge state of the battery for charging 61.

[0109] The battery for charging 61 is of a ‘battery pack’ configurationcomprised of plural nickel cadmium battery cells packed into acartridge.

[0110] The charging/discharging controller 62 measures the terminalvoltage or the charging/discharge current of the battery 61 and thetemperature around the battery 61 to grasp its residual capacity todetermine the start and end time points of charging. The charging startand end timings, as determined by the charging/discharging controller62, are notified to the controller 20 for use as a trigger for themobile robot 1 to start or terminate the charging operations.

[0111] The controller 20 is equivalent to a ‘brain’ and is loaded on,for example, the head unit 3 or the trunk unit 2 of the mobile robot 1.

[0112]FIG. 3 shows a more detailed structure of the controller 20. Asshown therein, the controller 20 is comprised of a CPU (centralprocessing unit) 21, as a main controller, interconnected over a buswith respective circuit components, such as memories, or peripheralcircuits. The bus 27 is a common signal transmission path, inclusive ofdata buses, address buses and control buses. To each device on the bus,there is assigned a unique address (memory address or I/O address), sothat, subject to addressing, the CPU 21 is able to communicate with aspecified device on the bus 28.

[0113] A RAM (random access memory) 22 is a write memory, comprised of avolatile memory, such as DRAM (dynamic RAM), and is used for loadingprogram codes, executed by the CPU 21, and for transient storage of workdata by a program then running.

[0114] A ROM (read-only memory) 23 is a read-only memory for permanentlyrecording programs or data. Among the program codes, stored in the ROM23, there are, for example, a self-diagnosis test program, executed onpower up of the mobile robot 1, and an operation control program forregulating the movements of the mobile robot 1.

[0115] Among the control programs for the robot 1, there are a ‘sensorinput processing program’ for processing sensor inputs, such as inputsfrom the camera 15 or the microphone 16, a ‘behavior command program’for generating the behavior, that is movement patterns, of the mobilerobot 1 based on the sensor inputs and a preset movement model, and a‘driving control program’ for controlling the driving of respectivemotors or speech outputs of the loudspeaker 17 in accordance with the sogenerated movement pattern. The generated movement pattern may include,in addition to the routine walking and running movements, the speech ormovements rich in entertainment properties, such as [ote] (meaning‘raise hands’), [oazuke] (meaning ‘raise fore feet’, [osuwari] meaning‘sit down’, or whining such as [waw-waw].

[0116] Among the control programs for the robot 1, there are a varietyof programs for operating sequences, created and edited using anauthoring tool. The authoring tool is booted in an environment in whicha preset program is run on a computer system installed outside the robot1. The authoring tool and the programs created and edited on the toolwill be explained subsequently.

[0117] The non-volatile memory 24 is constructed by electricallyerasable and overwritable memory devices, such as EEPROM (electricallyerasable programmable ROM), and is used for non-volatile storage of datawhich should be updated sequentially. Among the data to be updatedsequentially, there may, for example, be the security information, suchas production numbers or secret keys, and a variety of models regulatingthe behavioral pattern of the robot 1.

[0118] An interface 25 is a device for establishing interconnection withequipment other than the controller 20 to enable data exchange. Theinterface 25 effectuates input/output of data with e.g. the camera 15,microphone 16 or the loudspeaker 17. The interface 25 also effectuatesinput/output of data and commands with respective drivers 53-1 . . .within the driver 50.

[0119] The interface 25 may also be provided with serial interfaces,such as RS (Recommended Standard)-232C, parallel interfaces, such asIEEE (Institute of Electrical and Electronics Engineers) 1284, orgeneral-purpose interfaces for connection to computer peripherals, suchas USB (Universal Serial Bus) interface, i-Link (IEEE1394) interface,SCSI (small computer system interface) or memory card interface, to runthe programs or to effect data transfer with locally connected externalequipment.

[0120] By way of another instance of the interface 25, an interface forIR communication (IrDA) may be used to effectuate radio communicationwith external equipment.

[0121] The controller 20 may also be provided with a radio communicationinterface 26 and a network interface card (NIC) 27 to effectuateproximity radio communication, such as ‘Bluetooth’ or ‘11B’, or datacommunication with the external host computer 100 over LAN (Local Areanetwork, e.g., Ethernet) or Internet. The transmission/receipt unit forradio communication is preferably provided on distal ends of the mainbody unit of the mobile robot 1, such as head unit 2 or tail 3, in lightof reception sensitivity.

[0122] An objective of data communication between the mobile robot 1 andthe host computer 100 is to compute complex operational control of themobile robot 1, or to perform remote control, using computer resources,that is resources outside the robot 1, that is those on a remotecomputer.

[0123] Another objective of the data communication is to supply data orprograms, necessary for program codes, such as movement model, necessaryfor movement control of the robot 1, from a remote device to the mobilerobot 1 over the network.

[0124] Still another objective of the data communication is toeffectuate downloading of the robot movement controlling program,created and edited on the host computer 100 using an authoring tool, aslater explained, and debugging in real-time of the operation controllingprogram by cooperative operation in concert between the host computer100 and the robot 1.

[0125] Yet another objective of the data communication is to transferoperational setting values, such as angular data, of respective jointactuators, prescribing the pose directly taught by the operator to themobile robot 1, to the host computer 100. On the host computer 100, the‘pose’ prescribed by the angular data of the respective joint actuatorsmay be edited on the pose window to produce a key frame for motion. Inshort, the pose provided from the mobile robot 1 may be used for e.g.,action editing.

[0126] The controller 20 may be provided with a keyboard 29, made up often keys and/or an alphabetical keys. The keyboard 29 is directly usedfor inputting a command by the user on the operating site of the robot1, while being used for inputting the owner authentication information,such as a password.

[0127] The mobile robot 1 of the instant embodiment is able to executeautonomous movements, that is movements devoid of human intervention, bythe controller 20 executing a preset movement controlling program.Moreover, the mobile robot 1 of the instant embodiment is provided witha picture inputting device, that is camera 15, a speech input device,that is microphone 16, or an input device equivalent to the five sensesof the human being or animals, such as touch sensor 18, while beingendowed with intelligence sufficient to enable intellectual or emotionalmovements to be executed responsive to these external inputs.

[0128] The mobile robot 1, constructed as shown in FIGS. 1 to 3, has thefollowing characteristics:

[0129] (i) When commanded to transfer from a given orientation toanother orientation, the mobile robot 1 is able to effectuate transitionwithout transition directly through the respective orientations, butthrough unforced intermediate orientations, as provided from the outset.

[0130] (ii) When an optional orientation is reached through transitionthrough preset orientations, a notice to that effect may be received.

[0131] (iii) The orientation control may be managed as the orientationis supervised independently from one unit, such as head or leg or tailunit, to another. That is, unit-based orientation control may be managedfrom unit to unit, apart from the orientation control of the robot 1 inits entirety.

[0132] (iv) It is possible to deliver parameters for specifying detailedoperations of the movement commands.

[0133] Referring to FIG. 3, the mobile robot 1 of the instant embodimentis connected over a network to the external host computer 100.Alternatively, the mobile robot 1 may be connected to the host computer100 by radio communication, such as Bluetooth or 11B proximity radiocommunication, or the like communication means.

[0134] On the host computer 100, a preset software execution environmentis constructed, so that, under this environment, the authoring tool canbe booted to create and edit the operating sequence of the robot 1 byinteractive processing relatively easily and efficiently. The authoringtool will be discussed in detail subsequently.

[0135]FIG. 4 schematically illustrates an exemplary structure of thehardware of the host computer 100. The respective components of thecomputer 100 will be hereinafter explained in detail.

[0136] A CPU (central processing unit) 101, as a main controller of thesystem 100, is configured for executing various applications undercontrol by an operating system (OS). Preferably, the OS furnishes a GUI(graphical user interface) environment. For example, the OS may beWindows98/NT of Microsoft Inc., USA.

[0137] As shown, the CPU 101 is connected over bus 107 to otherequipment, not shown. The respective equipment on the bus 107 areafforded with unique memory addresses or I/O addresses, so that the CPU101 is able to access the equipment based on these addresses. The bus107 is a common signal transmission path, including data, address andcontrol buses, and may be typified by a PCI (Peripheral ComponentInterconnect) bus.

[0138] A memory 102 is used for storage of program codes executed on theCPU 101 or transient storage of current operating data. It should benoted that the memory 102 shown may be a non-volatile memory or avolatile memory, as desired.

[0139] A display controller 103 is a dedicated controller for actuallyprocessing drawing commands issued by the CPU 101, and supports thebitmap drawing function equivalent to, for example, SVGA (Super VideoGraphic Arrays) or XGA (eXtended Graphic Array). The drawing data,processed by the display controller 103, is transiently written in aframe buffer, not shown, and subsequently output to a display device111, which display device 111 may, for example, be a CRT (Cathode RayTube) display or a liquid crystal display.

[0140] An input equipment interface 104 is a device for connecting userinput equipment, such as keyboard 112 or mouse 113, to the system 100.The input equipment interface 104 is responsive to a coordinate commandinput through key input from a keyboard 112 or a mouse 113 to produceinterrupt in the CPU 101.

[0141] A network interface 10S is able to connect the system 100 to anetwork, such as LAN (Local Area network), or to a proximity radio datacommunication, such as Bluetooth or 11B, in accordance with a presetcommunication protocol, such as Ethernet. The network interface 10S isfurnished in general in a form of a LAN adapter card, and is used as itis loaded in a PCI bus slot on a motherboard, not shown.

[0142] In the embodiment shown in FIG. 3, the host computer 100 isinterconnected to the robot 1 via radio data communication or a network.Of course, the computer 100 may also be connected to the robot 1 byother suitable communication means or data movement means. For example,data exchange or movement may also be performed using a recordingmedium, such as memory cards (memory stick).

[0143] On the network, plural host computers, not shown, areinterconnected in a transparent state to form a distributed computerenvironment. On the network, software programs or data are distributed.For example, the authoring tool of the instant embodiment and themovement sequence program for robots, prepared and edited by the presentauthoring tool, or even the action files, motion files, sound files orLED operation files, as the movement files, may be distributed over thenetwork. The network distribution service of these programs/data may berendered either for pay or free of charge.

[0144] An external equipment interface 106 is a device for connecting anexternal device, such as a hard disc drive (HDD) 114 or a media drive115, to the system 100. The external equipment interface 106 is designedso as to comply with interface standards, such as IDE (Integrated DriveElectronics) or SCSI (Small Computer System Interface).

[0145] The HDD114 is an external storage device, well-known per se,having a magnetic disc as a storage medium loaded thereon permanently,and is superior to other external storage devices as to storage capacityand data transfer rate. The mounting of a software program on the HDD116 in an executable state is termed installation of the program on asystem. In the HDD 114, program codes of the operating system to beexecuted by the CPU 101, application programs or device drivers arestored in a non-volatile fashion. For example, the authoring tool of theinstant embodiment, or the movement sequence program, prepared andedited using the authoring tool, may be installed on the HDD 114.

[0146] On the other hand, a media drive 115 is a device in which a CD(Compact Disc), MO (magneto-optical disc) or a DVD (Digital VersatileDisc) is loaded for accessing the data recording surface. A portablemedium 115 a is used for backup mainly of software programs or datafiles in a computer readable data form or for moving the data betweendifferent systems, such as selling, circulating or distributing thedata. For example, the authoring tool of the instant embodiment and themovement sequence program for robots, prepared and edited by the presentauthoring tool, or even the action files, motion files, sound files orLED operation files, may be physically circulated or distributed betweendifferent equipment, using the portable medium 115 a.

[0147] Meanwhile, an example of the host computer 100, shown in FIG. 4,is a computer compatible with or inheriting a personal computer “PC/AT(Personal Computer/Advanced Technology)” manufactured by IBM of USA. Ofcourse, computer systems equipped with other architectures may be usedas the host computer 100 of the instant embodiment.

[0148] B. Structure of Authoring System

[0149] With the instant embodiment, the movement control program,comprised of a series of command/data, stating a preset movement patternof the robot 1, may be created and edited, using an authoring toolbooted on the computer 100. The movement control program, prepared andedited using this authoring tool, may be transferred to the robot 1,using radio communication means, such as Bluetooth or 11B, to executedebugging by concerted operation between the host computer 100 and therobot 1. That is, the authoring system for supporting the creation andediting of the movement control program is constructed by organiclinkage between the host computer 100 and the robot 1.

[0150]FIG. 5 schematically shows the overall structure of the authoringsystem.

[0151] On the host computer 100, the user is able to prepare and edit aprescribed scenario for the mobile robot 1, by a mouse operation, usingthe GUI picture presented by the authoring tool. The GUI picture forpreparing the scenario and the editing operation on the GUI picture willbe explained in detail subsequently. Alternatively, the user is able tocreate and edit the movement control program for the robot 1, in ascript form, such as a high grade language system, e.g., C-language,using a routine text editor etc.

[0152] The authoring tool converts the scenario, prepared and edited bythe user on the GUI picture, or the movement control program of thescript style, prepared and edited on the text editor, into anassembler-like pneumonic code, termed “RCODE”.

[0153] The RCODE means a program language, formulated to control therobot 1 by simple commands. Since the RCODE also has a basic controlstructure, such as “IF” or “GO”, it may also be used as a the lowestlevel script language for robot control.

[0154] The RCODE movement control program, created and edited on theside host computer 100, may be moved to the robot 1 using e.g., astorage medium, such as memory stick. In debugging the RCODE movementcontrol program, the RCODE program is taken out row by row and encryptedso as to be sequentially transferred to the robot 1, using radiocommunication means, such as Bluetooth or 11B.

[0155] The robot 1, on the other hand, is provided with aninterpreter/debugger, middleware, driver and an operating system (OS),as an environment for execution and debugging of the movement controlprogram, stated in the RCODE etc.

[0156] The interpreter is a high level language program for reading-in,row-by-row, the program stated in the RCODE form, for executing the soread-in program. It should be noted that, when the RCODE program in theencrypted form is transmitted from the side host computer 100, as indebugging, it is necessary for an interpreter to decode it transientlybefore proceeding to interpretation and execution.

[0157] The debugger is a program for finding and correcting errors(bugs) in the RCODE program. That is, a debugger is able to halt theprogram running at a specified row or to refer to the contents of thememory or the variable at such time.

[0158] The middleware is a set of software modules furnishing basicfunctions of the robot 1. The structure of each module is affected bymechanical and electrical properties of the robot 1, or hardwareattributes, such as specifications or shape. Functionally, themiddleware is classified into the middleware of the recognition systemand that of the output system.

[0159] The middleware of the recognition system is an engine forreceiving and processing crude data from the hardware, such as pictureor speech data or detection data obtained from other types of thesensor, through a virtual robot. That is, the middleware of therecognition system performs speech or color recognition or detection ofdistance, orientation, contact, motion or color to acquire the resultsof recognition, which are notified to an upper application layer, thatis a movement control program.

[0160] On the other hand, the middleware of the output system furnishesthe functions, such as reproduction of the walking and movements,synthesis of output sound or control of the turning on/off of LEDindicators. That is, the middleware of the output system receivesbehavioral schedules, mapped out on the application layer, to generatethe servo control values of the robot, output sound, output light (LED)or output speech, on the joint basis, from one function of the robot 1to the next, to realize the various actual performances on the robot 1.

[0161] The driver is a program code for performing operations of therespective joint actuators and other hardware operations.

[0162] In the present embodiment, the middleware and the driver areimplemented by the object oriented programs. The object orientationbased software is handled in terms of a module termed an ‘object’ whichbasically is data and a processing sequence for the data, unifiedtogether. One software is completed by preparing or combining pluralobjects as necessary. By and large, the object oriented programming isfelt to help improve software development and maintenance.

[0163] An operating system (OS) exercises control of data communicationbetween these objects and of execution of other programs. The OS is alsoimplemented by an objet oriented program.

[0164] C. Creation and Editing of Operating Program for Robots EmployingAuthoring Tool

[0165] The movement scenario, created using the authoring tool of theinstant embodiment, is made up of creation and editing of the ‘behavior’and creation and editing of the ‘action’, with the resultant productbeing termed a ‘project’. In this project is set the configuration orCPC (configured peripheral component) of the mobile robot 1, that is thehardware constituent information, comprised of the combination of thetrunk, head and legs of the mobile robot 1.

[0166] The project is made up of the behavior file, action file, motionfile, sound file and LED operation file. The behavior is composed ofcombination of actions. Each action is made up of respective contents,such as motion, sound and the LED operations.

[0167] The motion file is a file which prescribes the movements of therespective joint actuators of the mobile robot 1. In the instantembodiment, the motion is prescribed by chronologically arranging two ormore key frames delineating the mobile robot 1 striking a pose on a GUIediting picture. The motion editing operation on the GUI editing picturewill be explained later in detail.

[0168] The sound file is acoustic data for speech outputting vialoudspeaker 17, and is constructed e.g., as MIDI (Musical InstrumentalDigital Interface) or WAVEform file. For example, the acoustic file,stated in the MIDI form, is designed to represent music not as theinformation of the sound itself, but on converting the performanceinformation, such as loudness, length, timbre or effect into numericaldata. In the instant embodiment, the numerical data of the MIDI form,constituting the sound, are acted on through the GUI editing picture forediting the performance information, as will be explained subsequentlyin detail.

[0169] The LED movement file is data for prescribing the combination ofthe turning on/off of the plural LED indicators 19, and is used forforming facial expressions. In the instant embodiment, the LED operationfile is stated in the MIDI form, and is designed to be edited in desiredmanner through the GUI editing picture. The editing operation of the GUIediting picture will be explained in detail subsequently.

[0170] The motion, sound and the LED operations represent constituentelements of an action, and are chronological data changing with lapse oftime. For accurate reproduction of an action, these constituent elementsmust be temporally synchronized with one another. In the presentembodiment, each file can be edited so that the motion and the sound aswell as the turning on/off of the plural LEDs will be synchronized withone another on the time axis, as will be explained in detailsubsequently.

[0171] An action is constructed by unifying the respective contents,that is the motion file, sound file and the turning on/off of the pluralLEDs. One action file is a command reproduced in approximately tenseconds, and is also termed ‘semantics’. In the present embodiment,there is provided a working environment for action edition in which timelines are utilized on the GUI editing picture to achieve thesynchronization between the respective contents with much ease, as willbe explained subsequently. The respective contents can not only beprocessed as individual data, but may be handled as an action, that isin a form unified with other contents.

[0172] The behavior is a file prescribing the behavior of the mobilerobot 1 and which is constituted by arraying two or more commands, thatis actions. An action is reproduced in one direction from the beginninguntil the end. Conversely, with the behavior, the action reproducingsequence can be prescribed. In addition, subroutines may also be definedby branching based on conditions or probability or by forming pluralcommands or actions into a box. Thus, with the behavior, it is possibleto state a higher and more complex movement sequence of the mobile robot1 than with the action.

[0173]FIG. 29 schematically shows a functional structure of theauthoring system. The authoring system of the present embodiment, showntherein, is particularly designed for editing an action, and is made upby an action editor, a keyframe editor, a motion editor, a sound editor,an LED movement editor and a user interface controller for achieving theediting operation by the user with these functional modules by dialog onthe GUI picture.

[0174] The action editor is a functional module for editing the motionfile, the sound file and the LED movement file with synchronization onthe time axis. The action editor presents, through the user interfacecontroller, the joint movement (motion) along the time axis of themobile robot 1, and an action editing window, for setting the timing ofthe sound and the LED operations, to the user. The action editingwindow, including an editing area, made up by a time-line style tablefor setting the various files on the time axis, will be explained indetail subsequently.

[0175] The keyframe editor is a functional module for editing akeyframe, that is a picture frame delineating a pose of the mobile robotperforming a motion at a relevant time point. The key frame editor isinvoked responsive to the user's actuation on the action editor andaccepts the editing operation by the user through a keyframe channelopened on the action editing window. The keyframe channel, in whichthumbnails representing the keyframe are placed at relevant positions onthe time axis, will be explained im detail subsequently.

[0176] The motion editor is a functional module for editing the motion,that is chronological movements of the respective joint actuators makingup the mobile robot. The motion editor is invoked responsive to theuser's actuation on the action editor and accepts the editing operationby the user through a motion channel opened on the action editingwindow. The motion channel, in which each timing chart stating thechronological movements of the respective joint actuators is listed up(in a tree-view) in a living body simulating model, will be explained indetail subsequently.

[0177] The motion editor presents, through the user interfacing unit,the pose window for editing the pose of the mobile robot 1 on a 3Ddisplay screen, and a motion previewer, for pre-viewing the finishedmotion, to the user. The pose window and the motion previewer will beexplained in detail subsequently.

[0178] The sound editor is a functional module for setting details ofthe sound as one of the constituent elements of an action. In thepresent embodiment, the sound is treated in the MIDI or WAVE form. Thesound editor presents to the user the sound detail window, usable forsetting details of the sound along the time axis, through user interfacecontroller. The sound detail window includes an editing area, comprisedof a two-dimensional time-line style table, made up by a time axis inthe horizontal direction and by a channel in the vertical direction. Thesound detail window will be explained later in detail. The contents asset on the sound detail window are used for demonstration on a soundchannel, as later explained, within the action editing window.

[0179] The LED movement editor is a functional module for settingdetails of the LED operation as one of the constituent elements of theaction. In the present embodiment, the LED movement is treated in theMIDI style. The LED movement editor presents, through the user interfacecontroller, an LED detail window usable for setting details of the LEDmovements along the time axis, to the user. The LED detail window,including an editing area, comprised of a two-dimensional time-linestyle table, made up by a time axis in the horizontal direction and by achannel in the vertical direction, will be explained subsequently. Thecontents as set on the LED detail window are used for demonstration onan LED movement channel, as later explained, within the action editingwindow.

[0180] In editing the project, the user interface controller presents aproject window to the user.

[0181] The user interface controller is responsive to the user command,input through each editing window, to access each file system ordatabase supervising the behavior file, action file, motion file, soundfile and the LED movement file.

[0182] The sequence of processing operations for a user to create andedit the movement scenario of the mobile robot 1 on the authoring systemof the present invention is now explained in detail.

[0183] In editing the project, a ‘project window’ shown in FIG. 6 isdemonstrated. The project window includes a title bar, a menu bar, atool bar and a list display area. The project window is formed e.g., byan SDI (Single Document Interface) main window. The SDI is a userinterface of the type operating for opening one document in one window.

[0184] As shown, a list of files, namely the behavior file, action file,motion file, sound file and the LED movement file, used for editing thebehavior and the action, is displayed in a tree style in the projectwindow.

[0185] If a listed item of the behavior or the action is double-clicked,the relevant editing window, explained later with reference to FIG. 9,is opened. If the thumbnail of the lower left motion is double-clicked,the motion preview window, later explained with reference to FIG. 28, isopened.

[0186] The project window has the function of displaying the detailedinformation concerning the item as selected on the tree list.

[0187] The detailed information concerning the behavior includesthumbnails, filenames, items of classification, epitomized descriptionof operation and operating time. The detailed information concerning theaction includes the thumbnail (initial pose), filename, operating time,initial pose, end pose or the file capacity. The detailed informationconcerning the motion includes the thumbnail (initial pose), filename,operating time, initial pose, end pose and the file capacity. Thedetailed information concerning the sound includes the thumbnail (iconrepresenting the sound), filename, operating time and the file capacity.The detailed information concerning the LED operation includes thethumbnail (icon representing the sound), filename, operating time andthe file capacity.

[0188] On top of the project window, there are provided a menu bar,including menus ‘file’, ‘edit’, ‘material’ and ‘help’.

[0189] If the menu ‘file’ is selected, a pull-down menu, made up ofrespective sub-menus ‘new project’, ‘open project’, ‘save project’,‘save new project’ and ‘end’, appears (see FIG. 7).

[0190] If the sub-menu ‘new project’ is selected, a new project isproduced. If an unsaved project is already opened, a dialog promptingthe user to confirm whether or not the project is to be saved appears.

[0191] If the sub-menu ‘open project’ is selected, the pre-existingproject file is opened. If the unsaved project is already opened, adialog appears to prompt the user to confirm whether or not the projectis to be saved, in the manner as described above.

[0192] If the sub-menu ‘save project’ is selected, the relevant projectfile is overwrite-saved. If the file has not been saved, a file settingdialog, as later explained, appears, as in saving a new project, toprompt the user to input the filename.

[0193] If the sub-menu ‘save new project’ is selected, a file settingdialog appears to prompt the user to input the filename.

[0194] If the sub-menu ‘end’ is selected, this project window is closed.If the project file in the window has not been saved, a dialog appearsto prompt the user whether or not this is to be saved.

[0195] If the menu ‘material’ is selected, a pull-down menu, made up ofsub-menus ‘create new behavior’, ‘create new action’, ‘read-in material’and ‘delete material’ appears (see FIG. 8).

[0196] If the sub-menu ‘create new behavior’ is selected, the behaviorediting window is opened in a novel state. This behavior isautomatically added to the list of the behavior in the project window.The behavior editing is not directly related to the purport of thepresent invention and hence is not explained in further detail herein.

[0197] If the sub-menu ‘create new action’ is selected, the actionediting window 200, as later explained and as shown in FIG. 9, is openedin a novel state. This action is automatically added to the list of theactions in the project window.

[0198] If the sub-menu ‘read-in material’ is selected, a file specifyingdialog appears to register a usable material file in the project. Thisoperation is the same as that occurring on drag-and-drop from theexplorer.

[0199] If the sub-menu ‘delete material’ is selected, valid displayoccurs only when the item is in a selected state. On selection, the itemis deleted from the list in the project window. It should be noted thatit is not the file that is deleted from the directory.

[0200] The menu ‘help’ includes sub-menus such as topic, support web orversion information.

[0201] Below the menu bar, there are mounted tool buttons forinstantaneously invoking frequently used functions, such as create newbehavior button (New Behavior), create new action button (New Action) ortrash (Trash).

[0202] The create new behavior button is equivalent to the sub-menu‘create new behavior’ in the menu ‘material’. The create new actionbutton is equivalent to the sub-menu ‘create new action’ in the menu‘material’. The trash is equivalent to the sub-menu ‘delete material’ inthe menu ‘material’.

[0203] In the project window, the so-called drag-and-drop operation isallowed. That is, a file icon on the explorer may be dragged to anoptional site of the tree for direct registration.

[0204] Moreover, the material on the tree list can be dragged to anydesired editing window. The action can be dragged to the behaviorwindow, whilst the motion, sound and LED operations can be dragged totthe action window.

[0205]FIG. 9 schematically shows the structure of an action editingwindow 200. On this action editing window 200, the timings of the jointmovement (motion) and the those of the sound and LED operations of themobile robot 1 along time axis may be set. The results of editing on theediting window are saved as action files having the extension “act”.Meanwhile, in the case of the authoring system in which different actionediting windows 200 are provided depending on the difference in themodel of the mobile robot 1 (or on the difference of the information onthe hardware structure comprised of the combination of CPC components),the action editing windows 200 may be adapted to be switched responsiveto the model selection operations by the user.

[0206] As shown, the action editing window 200 is made up of a titlebar, a menu bar and an editing area for setting the motion data, sounddata and the LED operation data on the time axis.

[0207] In the menu bar, there are provided menus ‘file’, ‘material’ and‘help’.

[0208] If the menu ‘file’ is selected, there appears a pull-down menucomprised of sub-menus ‘new action’, ‘open action’, ‘save action’, ‘svenew action’, ‘replay on actual machine’ and ‘close’ (see FIG. 10).

[0209] If the sub-menu ‘new action’ is selected, a novel action isproduced. If already an unsaved action is opened, there appears a dialogprompting the user to confirm whether or not the action is to be saved.

[0210] If the sub-menu ‘open new action’ is selected, a pre-existingaction file is opened. If an unsaved action has already been opened,there appears a dialog prompting the user to confirm whether or not theaction is to be saved.

[0211] If the sub-menu ‘save action’ is selected, the relevant actionfile is overwrite-saved. In case of an unsaved file, a file settingdialog appears, as in the case of new action saving, as later explained,to prompt the user to input the filename.

[0212] If the sub-menu ‘save new action’ is selected, a file settingdialog appears to prompt the user to input a filename.

[0213] If the sub-menu ‘replay on actual machine’ is selected, theaction file, edited on the action editing window 200, shown in FIG. 9,is transferred to the actual machine, that is to the mobile robot 1, toreally reproduce the file by way of a trial. The action file may betransferred to the mobile robot 1 via near radio data communication,such as Bluetooth, or moved between devices through the intermediary ofa medium, such as a memory card.

[0214] If the sub-menu ‘close’ is selected, this action editing window200 is closed. If an action file(s) in the window is as yet not saved, adialog appears to prompt the user to confirm whether or not this file isto be saved.

[0215] If the menu (edit) is selected, there appears a pull-down menucomprised of sub-menus ‘return’, ‘slice’, ‘copy motion’, ‘copy sound’,‘copy LED’. ‘overwrite paste’, ‘insert paste’, ‘delete’, ‘formkeyframe’. ‘insert frame’, ‘insert number of specified frames’, ‘deleteframes’ and ‘delete specified frame’ (see FIG. 11).

[0216] If the sub-menu ‘revert’ is selected, undo processing is carriedout beginning from the near-by operation.

[0217] If the sub-menu ‘slice’ is selected, the motion over a selectedtime span, if any, is cut, whereas, if there is no such time span, themotion, sound and the LED operations of the frame in question, are cut.Actually, the so cut data is transiently stored in a clipboard. By thiscutting, not the frame itself but the information concerning thecontents of the frame is lost.

[0218] If the sub-menu ‘copy motion’ is selected, the motion over aselected time span, if any, is cut. Actually, the so cut data istransiently stored in a clipboard.

[0219] If the sub-menu ‘copy sound’ is selected, the sound over aselected time span, if any, is cut. Actually, the so cut data istransiently stored in a clipboard.

[0220] If the sub-menu ‘copy LED’ is selected, the LED operation over aselected time span, if any, is copied. Actually, the so copied data istransiently stored in a clipboard:

[0221] If the sub-menu ‘overwrite paste’ is selected, the contentsstored in the clipboard are pasted at the current time point.

[0222] If the sub-menu ‘insert paste’ is selected, the contents storedin the clipboard are inserted and pasted at the current time point.

[0223] If the sub-menu ‘delete’ is selected, the motion over a selectedtime span, if any, is deleted, whereas, if there is no such time span,the motion, sound and the LED operation, that is the action, of theframe in question is deleted.

[0224] If the sub-menu ‘form keyframe’ is selected, the frame of thecurrent time point is formed into a key frame. That is, a pose can begenerated on interpolation between pre-existing keyframes and formedinto new keyframe data.

[0225] If the sub-menu ‘insert frame’ is selected, the frame(s) for aselected time span, if any, is inserted at the current time point. As aresult of the frame insertion operation, the overall time of the actionis elongated by the specified frame time period.

[0226] If the sub-menu ‘insert specified number of frames’ is selected,there appears a numerical input dialog so that a number of framescorresponding to the numerical value as input on the dialog areinserted. As a result of the insertion of the specified number offrames, the overall time of the action is elongated by the specifiedframe time period.

[0227] If the sub-menu ‘delete frame’ is selected, the frame(s)corresponding to a selected time span, if any, is deleted from thecurrent time point. At this time, frames lying towards left in theediting area are padded. As a result of the frame deletion, the overalltime of the action is shorter by the time corresponding to the deletedframes.

[0228] If the sub-menu ‘delete specified frames’ is selected, thereappears a numerically input dialog to delete the frames corresponding tothe numerical figure input on the dialog. The numerical figures as inputare time-specifying figures. At this time, frames lying towards left inthe editing area are padded. As a result of the frame deletion, theoverall time of the action is shorter by the time correspondingspecified frame time.

[0229] If the menu ‘material’ is selected, there further appears apull-down menu, comprised of ‘read-in motion’, ‘write-out motion’,‘read-in sound’, ‘write-out sound’, ‘read-in LED’ and ‘write-out LED’(see FIG. 12).

[0230] If the sub-menu ‘read-in motion’ is selected, the motion file isread-in from the specified site of storage, such as local disc, so as tobe inserted at the current time point on a motion channel 203 in theediting area as later explained. During this read-in operation, thekeyframe included in the motion file directly becomes a keyframe.

[0231] If the sub-menu ‘write out motion’ is selected, the motion of theselected time span is stored as the motion file in the site of storagespecified, such as local disc.

[0232] If the sub-menu ‘read-in sound’ is selected, the sound file isread-in from the site of storage as specified, such as local disc, andis inserted at the current time point on a sound channel 204 in theediting area.

[0233] If the sub-menu ‘write-out sound’ is selected, the sound of thespecified time span is stored as a sound file of, for example, the MIDIstyle, in the specified storage site, such as local disc.

[0234] If the sub-menu ‘read-in LED’ is selected, the LED operating fileis read-in from the site of storage as specified, such as local disc,and is inserted at the current time point on an LED channel in theediting area, as later explained.

[0235] If the sub-menu ‘write-out sound’ is selected, the sound of thespecified time span is stored as an LED file of, for example, the MIDIstyle, in the specified storage site, such as local disc.

[0236] In the menu ‘help’, there are provided sub-menus, such as topics,support webs and the version information.

[0237] Reverting to FIG. 9, the editing area in the action editingwindow 200 is explained. As shown, the editing area of the actionediting window 200 is a two-dimensional time line style table comprisedof the time axis in the horizontal direction and the channel in thevertical direction. Within the time line table, there are a time ruler201, a keyframe channel 202, a motion channel 203, a sound channel 204and an LED operation channel 205.

[0238] With the time ruler 201, real-time display can be switched todisplay of the number of frames, and vice versa, using a unit switchingradio button 207. In the instance shown in FIG. 9, real-time display isselected. The real-time display is graduated in seconds andmilliseconds, each being of two digits. The relationship between displayin real-time display and display on the time ruler 201 is shown by thefollowing equation 1:

[0239] (Equation 1)

00:00=0 sec

15:23=5 sec 23

13:87=13 sec 87.

[0240] In the graduation of the time ruler 201 in the representation inthe number of frames, the number of frames is in four digits. Themaximum number of frames of 9999 is approximately 160 seconds.

[0241] The frame width setting on the picture in each case of the realtine display in second and display in the number of frames, with theunit being f, is as shown in the following Tables 1 and 2: TABLE 1 RealTime Display possible display time reading of graduations setting value(display) (seconds) (numerical figure) auxiliary graduations 1 frame =0.5 pixel (0.5 px/fr) ca. 20.12 seconds 02:00 (each 62.5 px) 00:20 (each6.25 px) 1 frame = 1 pixel (1 px/fr) ca. 10.24 seconds 01:00 (each 62.5px) 00:10 (each 6.25 px) 1 frame = 2 pixel (2 px/fr) ca. 5.12 seconds01:00 (each 125 px) 00:10 (each 12.5 px) 1 frame = 3 pixel (3 px/fr) ca.3.41 seconds 00:50 (each 93.75 px) 00:05 (each 9.375 px) 1 frame = 4pixel (4 px/fr) ca. 2.56 seconds 00:50 (each 125 px) 00:05 (each 12.5px) 1 frame = 6 pixel (6 px/fr) ca. 1.7 seconds 00:25 (each 93.75 px)00:025 (each 9.375 px) 1 frame = 8 pixel (8 px/fr) ca. 1.28 seconds00.25 (each 125 px) 00:025 (each 12.5 px)

[0242] TABLE 2 Display in Number of Frames possible display time readingof graduations setting value (display) (seconds) (numerical figure)auxiliary graduations 1 frame = 0.5 pixel (0.5 px/fr) ca. 20.12 seconds200 (each 100 px) 20 (per 10 px) 1 frame = 1 pixel (1 px/fr) ca. 10.24seconds 100 (each 100 px) 10 (per 100 px) 1 frame = 2 pixel (1 px/fr)ca. 5.12 seconds 50 (each 100 px) 5 (per 10 px) 1 frame = 3 pixel (1px/fr) ca. 3.41 seconds 25 (each 75 px) 2.5 (per 7 px) 1 frame = 4 pixel(1 px/fr) ca. 2.56 seconds 25 (each 100 px) 2.5 (per 10 px) 1 frame = 6pixel (1 px/fr) ca. 1.7 seconds 10 (each 60 px) 5 (per 6 px) 1 frame = 8pixel (1 px/fr) ca. 1.28 seconds 10 (each 80 px) 1 (per 8 px)

[0243] Meanwhile, the possible display time in seconds means anapproximate number of seconds for the maximized window size in a displayin SVGA(Super Video Graphic Array). By the scrolling of the time axis(operation of a horizontal scroll bar 206, the time ruler 201 scrollsproperly.

[0244] The time ruler 201 includes an end time display field 208 and acurrent time display field 209, in addition to the unit switching radiobutton 207.

[0245] In the end time display field 208, a time representing numericalvalue, indicating the end time of an action being edited, that is theoperating time, is displayed (in the illustrated example, “09:40” (=9seconds 40), is displayed). In the current time display field 209, atime representing numerical value, indicating the current position, isdisplayed (in the illustrated example, “04:60” (=4 seconds 60) isdisplayed). These fields are editable text fields, such that, if ameaningful time representing numerical value is input, it becomes theend time, such that the final keyframe is moved or the current time ismoved to that position.

[0246] On the time ruler 201, an ‘initial pose specifying popup menu’,not shown, for specifying the initial pose, a ‘final pose specifyingpopup menu’, not shown, for specifying the final pose, and the ‘timewidth changing popup menu’, again not shown, for changing the intervalof the time ruler 201, may be invoked.

[0247] In the editing area, a ‘keyframe line 210’ an ‘end time line 211’and a ‘current time line 212’ are displayed.

[0248] A keyframe line 210, indicating the time of keyframes 202 a to202 b, as later explained, is displayed as intersecting the respectivechannels. Thus, the user is able to continue the editing operation asthe or she checks for synchronization between the motion, sound and theLED operations.

[0249] On the other hand, since the end time line 211, indicating theend time of an action being edited, is displayed as intersecting therespective channels, the user is able to visually grasp the time rangebeing edited. The end time line 211 may be grasped as the final posekeyframe line.

[0250] Moreover, the current time line 212, indicating the current time,is displayed as intersecting the respective channels. Basically, whenany one channel is clicked, the current time is moved to such clickedposition.

[0251] If the preview window, not shown, is opened in the editing area,a three-dimensional image of the mobile robot 1 at the current time isdisplayed at all times. This three-dimensional image is obtained by theauthoring system automatically generating an interpolated frame for thecurrent time and carrying out picture processing, such as coordinatetransformation, based on e.g., the movements of the respective jointactuators.

[0252] The keyframe channel 202 is an area in which to display thekeyframe along the time axis prescribed by the time ruler 201.

[0253] In the present embodiment, the keyframe channel 202 can be openedor closed as desired. FIG. 13 shows the action editing window 200 as thekeyframe channel 202 is opened (keyframe detail channel). On eachrelevant site of the time axis in the keyframe channel 202, there isplaced a thumbnail representing a key frame. What is termed the‘keyframe’ here means a picture frame delineating the pose at therelevant time of the mobile robot effectuating the motion.

[0254] The initial pose and the final pose means special keyframes atthe beginning and trailing ends of a keyframe channel. The first andlast keyframes are put from the outset.

[0255] During the time interval on the keyframe channel, devoid of akeyframe, a frame interpolated by the keyframes on both sides of thistime interval, referred to below as interpolated frame, is reproduced.The interpolated frame is not displayed on the keyframe. With theauthoring system of the instant embodiment, if the respective keyframesare arrayed on the keyframe channel, the motion smoothly interconnectingthe poses stated in the respective keyframes, is automatically generatedby computer processing. The center of gravity of the 3D model can alsobe set with each keyframe to approximate the apparent movements of the3D model to that of a real machine.

[0256] Each keyframe is arranged on the keyframe channel so that theleft end of the thumbnail will be the time point of the keyframe.Moreover, the thumbnail can be dragged in the left and right directionalong the time axis, the keyframe line then performing the followupmovement to change the time of the corresponding keyframe. It is notedthat the extension and contraction of the keyframe of the last pose isthe extension and contraction of the overall action time.

[0257] When the thumbnail is double-clicked, the pose editing window ofthe relevant pose is opened to enable the pose editing. It should benoted that the initial and trailing poses are not editable such thatdouble-clicking these poses cannot open the pose editing window. Thepose editing window will be explained in detail subsequently.

[0258] The motion channel 203 is an area for editing and displaying themotion contents along the time axis prescribed by the time ruler 201.

[0259] In the instant embodiment, the motion channel 203 may be openedand closed. FIG. 14 shows the action editing window 200, with the motionchannel 203 opened (motion detail channel). The motion is defined by themovements of the respective joint actuators forming the mobile robot 1.In the motion channel 203, the timing charts stating the chronologicalmovements of the respective joint actuators are listed up in a treestyle in accordance with the living body simulating model (tree view).

[0260] The segmented line graphs of the respective timing chartsrepresent time changes of movements, that is rotational angles, of therelevant joint actuators.

[0261] The setting values at the relevant time of the relevant jointactuators can be changed by vertically dragging the points ofintersection of the keyframe line 210 and the segmented line. Moreover,the dragging changes the poses corresponding to the keyframe line 210 toenable the automatic updating of the keyframe contents.

[0262] On the motion detail channel, the movement of each joint actuatorcan be visually grasped in the timing chart style. The segmented linecan be directly acted on by drag-and-drop. A segmented line concerning agiven joint actuator can be copied to another joint actuator. The resultis that the segmented line for a given joint actuator may be copied toanother joint actuator. Consequently, the motion file editing operationcan be performed intuitively, while the labor in the editing operationcan be saved appreciably. Moreover, by the copying operation, regularmovements, such as movements symmetrical in the left and rightdirection, may be edited extremely readily. There are also occasionswherein motion data of a given motion differs depending on thedifference of the model of the mobile robot 1, that is the difference inthe information in the hardware structure ascribable to differentcombinations of the CPC components. In such case, the data can bere-used as data for different models by applying data transformation onthe basis of the model-based fixed data.

[0263] The motion detail channel and its tree view are scrolledhorizontally and vertically in keeping with the operation of the scrollbar for the horizontal direction and that for the vertical direction.

[0264] The sound channel is an area for displaying sound data along thetime axis prescribed by the time ruler 201. In the present embodiment,the ‘sound detail window’ independent from the action editing window 200may be opened to edit the sound data of the MIDI style on the window byGUI operation. The sound detail window will be explained in detailsubsequently.

[0265] The sound channel 204 is provided with a sound ON/OFF check box213. By marking the check box, the sound may be issued in reproduction.

[0266] The LED operation channel 205 is an area in which to display LEDoperating data along the tine axis prescribed by the time ruler 201. Inthe present embodiment, an ‘LED detail window 500’ independent from theaction editing window 200 may be opened to edit the LED operating dataof the MIDI style on the window by GUI operation. The LED detail window500 will be explained later in detail.

[0267] The LED operating channel 205 is provided with an LED ON/OFFcheck box 214. By marking the check box, the LED operation may beenergized in reproduction.

[0268] The action editing window 200 is able to accept the drag-and-dropoperation from the project window (see FIG. 6). That is, the respectivefiles, as constituent elements of an action, such as motion file, soundfile or the LED file, can be directly dragged and dropped from theproject window for registration in the action window extremely readily.

[0269] The action editing window 200 is also able to accept thedrag-and-drop operation from the MS Windows explorer. That is, therespective files, as constituent elements of an action, such as motionfile, sound file or the LED file, can be directly dragged and droppedfrom the explorer window for registration in the action window extremelyreadily. The files registered in the action window are simultaneouslyregistered in the project window.

[0270]FIG. 15 schematically shows the structure of a sound detail window300 used for editing the sound file of the MIDI style. As shown, thesound detail window 300 is made up by a title bar, a menu bar and anediting area in which to execute the editing of the sound file of theMIDI style by GUI operations.

[0271] In the menu bar, there are provided respective menus ‘file’,‘edit’, ‘set’ and ‘help’.

[0272] If the menu ‘file’ is selected, there appears a pull-down menucomprised of sub-menus ‘new sound’, ‘open sound’, ‘save sound’, ‘savenew sound’ and ‘close’ (see FIG. 16).

[0273] If the sub-menu ‘new sound’ is selected, a novel MIDI sound isproduced. If already an unsaved MIDI sound is opened, there appears adialog prompting the user to confirm whether or not the MIDI sound is tobe saved.

[0274] If the sub-menu ‘open sound’ is selected, a pre-existing MINIsound file is opened. If an unsaved MIDI sound has already been opened,there appears a dialog prompting the user to confirm whether or not theaction is to be saved (see the same figure).

[0275] If the sub-menu ‘save sound’ is selected, the relevant actionfile is overwrite-saved. In case of an unsaved file, a file settingdialog appears, as in the case of new action saving, as later explained,to prompt the user to input the filename.

[0276] If the sub-menu ‘save new sound’ is selected, a file settingdialog appears to prompt the user to input a filename.

[0277] If the menu (close) is selected, the sound detail window 300 isclosed. If the sound file in the window is not as yet saved, thereappears a dialog prompting the user to confirm whether or not theunsaved file is to be saved.

[0278] If the menu ‘edit’ is selected, there appears a pulldown menucomprised of respective sub-menus of ‘revert’, ‘slice’, ‘copy’,‘overwrite paste’, ‘insert paste’ and ‘delete’ (see FIG. 17).

[0279] If the sub-menu ‘revert’ is selected, undo processing is carriedout beginning from a near-by operation.

[0280] If the sub-menu ‘slice’ is selected, the sound over a selectedtime span, if any, is cut. Actually, the so cut data is transientlystored in a clipboard. By this cutting, not the frame itself but theinformation concerning the contents of the frame is lost.

[0281] If the sub-menu ‘copy’ is selected, the sound over a selectedtime span, if any, is cut. Actually, the so cut data is transientlystored in a clipboard.

[0282] If the sub-menu ‘overwrite paste’ is selected, the contentsstored in the clipboard are pasted at the current time point.

[0283] If the sub-menu ‘insert paste’ is selected, the contents storedin the clipboard are inserted and pasted at the current time point.

[0284] If the sub-menu ‘delete’ is selected, the motion over a selectedtime span, if any, is deleted. It is not the frame itself but theinformation on the contents of the frame that is lost.

[0285] If the menu ‘setting’ is selected, there further appears apulldown menu comprised of sub-menus ‘length of quarter notes’ and‘meter’ (see FIG. 18).

[0286] The sub-menu ‘length of a quarter note’ further has a sub-menu,not shown, and can be subdivided only for one fourths of the note. Thelength so set is displayed as a grid on the score along the time axisdirection.

[0287] The sub-menu ‘meter’ further has a sub-menu specifying therhythm. Based on the value, here set, there is delineated a linedetermining the meter on the time axis grid on the score.

[0288] In the menu ‘help’, there are contained the sub-menus, such astopic, support web or the version information.

[0289] Reverting to FIG. 15, the editing area of the sound detail window300 for the MIDI sound is explained. The editing area is atwo-dimensional time-line table, formed by the time axis in thetransverse direction and by the channel in the vertical direction. Thetime line table is made up by a time ruler 301, a keyframe channel 302,a score channel 303 and a velocity channel 304.

[0290] The time ruler 301 exploits a unit switching radio button 305 toswitch between real time display and the display with the number offrames. In the instance shown in FIG. 15, real-time display is selected.The real-time display is in second: millisecond, each being of twodigits. As for the relationship between the real time in the real-timedisplay and the display with the time ruler 301, reference is had to theequation 1 above. The time ruler 301 with display with the number offrames is graduated so that the number of frames is represented by fourdigits. The maximum number 9999 of frames is equivalent to approximately160 seconds.

[0291] As for the frame width setting on the picture in each case of thereal time display in seconds and the display with the number of framesf, reference is had to Tables 1 and 2 above.

[0292] The time ruler 301 includes an end time display field 306 and acurrent time display field 307, in addition to the unit switching radiobutton 305. In the end time display field 306, a time representingnumerical figure indicating the end time of the action being edited,that is the operating time, is displayed. In the illustrated instance,“09:40” (=9 seconds 40) is displayed. In the current time display field307, a time representing numerical figure for the current position isdisplayed. In the illustrated instance, “04:60” (=4 seconds 60) isdisplayed. These fields are editable text fields. When a numericalfigure for any meaningful time point is input, it becomes the end timesuch that the last keyframe is moved or the current time is moved tosuch position.

[0293] On the time ruler 301, a ‘time width changing popup menu’, notshown, for changing the interval of the time ruler 301, may be invoked.

[0294] In the editing area, a ‘keyframe line 308’, an ‘end time line309’ and a ‘current time line’ 310 are displayed as time display lines.The keyframe line 308, representing the time of each keyframe, as laterexplained, is demonstrated as intersecting the respective channels. Theuser is able to perform the operation of editing the MIDI sound as theor she visually checks for synchronization with the keyframe. Moreover,since the end time line 309 indicating the end time of the action beingedited is displayed as intersecting the respective channels, the user isable to visually comprehend the time range for editing. Moreover, thecurrent time line 310, indicating the current time, is displayed asintersecting the respective channels. Basically, when any optionalchannel is clicked, the current time shifts to such position.

[0295] In the keyframe channel 302, keyframe positions, as acquired fromthe action editing window 200 along time axis as specified by the timeruler 301, are displayed. However, in distinction from the case of theaction editing window 200 (see FIG. 13), the keyframe channel cannot beopened nor closed within the sound detail window 300.

[0296] The score channel is an area for editing the MIDI sound by GUIoperation, and is made up by a piano keyboard and a basic grid along thetime axis direction. It should be noted that the effective sound rangediffers depending on the model of the mobile robot 1.

[0297] On the piano keyboard, the maximum sound range allowed by thehardware specifications of the mobile robot 1 is demonstrated by imagedisplay of the piano keyboard. Alternatively, the reproducible soundrange may be displayed brightly, with the remaining areas beingdisplayed in gray. In the basic C key portion, the pitch of the absolutesound, such as C3 or C4, is displayed.

[0298] In the score part, the grid of time width of the quarter note asset is displayed. By the value as set by the meter, described above, thelines of the two grids, that is two beats, three grids, that is threebeats, or four grids, that is four beats, are emphasized.

[0299] On the score channel, a score is formed by the sound duration asthe basis of the time axis and by the pitch of piano keys. One mesh istermed a cell. The cell with the sound is colored. However, in a replaymodel for only one sound, no sound can be put in the different scales onthe same time axis.

[0300] If a void, that is uncolored, cell is clicked, the sound of aduration of the sound note mark as selected is put. If the sound ispresent at another pitch position at the same time, the sound isreplaced by the clicked pitch. If the cell where the sound is alreadypresent is clicked, the sound is removed.

[0301] On the left side of the keyboard are displayed marks of notes,such as a sixteenth note, eighth note, quarter note, a half note, awhole note, dotted eighth note, dotted quarter note and a dotted halfnote. These sound note marks have reciprocally exclusive selectionstates such that only one of them is necessarily selected. The selecteditem is varied on mouse click operations.

[0302] The velocity channel 304 is an area for demonstrating theintensity of the velocity from one sound to another. Although theinstance of FIG. 15 shows the sound intensity in a bar graph, it mayalso be shown by a segmented line graph. The sound intensity at eachnodal point can be adjusted by dragging the upper end of each bar in thebar graph. The maximum sound volume is set as default.

[0303] A ‘replay button’ for commanding the replay of the edited soundmay also be provided in the sound detail channel.

[0304]FIG. 19 schematically shows the structure of a sound detail window400 for displaying the sound file of the WAVE form. As shown, the sounddetail window 400 is made up of a title bar, a menu bar and an editingarea for editing the sound file of the WAVE form.

[0305] In the menu bar, there are provided menus ‘file’, ‘edit’ and‘help’.

[0306] If the menu ‘file’ is selected, there appears further a pull-downmenu comprised of sub-menus ‘open sound’, ‘save sound’, ‘save new sound’and ‘close’ (see FIG. 20).

[0307] If the sub-menu ‘open sound’ is selected, the pre-existing WAVEsound file is opened. If an unsaved sound has already been opened, thereappears a dialog prompting the user to confirm whether or not the WAVEsound is to be saved.

[0308] If the sub-menu ‘save sound’ is selected, the relevant WAVE soundfile is overwrite-saved. If the sub-menu ‘save new sound’ is selected, afile setting dialog appears to prompt the user to input a filename.

[0309] If the sub-menu ‘close’ is selected, this sound editing window400 is closed. If a sound file(s) in the window is as yet not saved, adialog appears to prompt the user to confirm whether or not this file isto be saved.

[0310] If the menu (edit) is selected, there appears a pull-down menucomprised of sub-menus ‘return’, ‘slice’, ‘copy’, ‘overwrite paste’,‘insert paste’ and ‘delete’ (see FIG. 21).

[0311] If the sub-menu ‘revert’ is selected, undo processing is carriedout beginning from the near-by operation.

[0312] If the sub-menu ‘slice’ is selected, the sound over a selectedtime span, if any, is cut. Actually, the so cut data is transientlystored in a clipboard. By this cutting, not the frame itself but theinformation concerning the contents of the frame is lost.

[0313] If the sub-menu ‘copy’ is selected, the motion over a selectedtime span, if any, is copied. Actually, the so copied data istransiently stored in a clipboard.

[0314] If the sub-menu ‘overwrite paste’ is selected, the contentsstored in the clipboard are pasted at the current time point.

[0315] If the sub-menu ‘insert paste’ is selected, the contents storedin the clipboard are inserted and pasted at the current time point.

[0316] If the sub-menu ‘delete’ is selected, the motion over a selectedtime span, if any, is deleted. The frame itself is not lost, but is in asilent state.

[0317] In the menu ‘help’, there are included sub-menus, such as topic,support web or the version information.

[0318] The editing area of the sound detail window 400 for WAVE sound istwo-dimensional time-line table, formed by the time axis in thetransverse direction and by the channel in the vertical direction. Thetime line table is made up by a time ruler 401, a keyframe channel 402and a WAVE channel 403.

[0319] The time ruler 401 exploits a unit switching radio button 404 toswitch between real time display and the display with the number offrames. In the instance shown in FIG. 15, real-time display is selected.The real-time display is in second: millisecond, each being of twodigits. As for the relationship between the real time in the real-timedisplay and the display with the number of frames, reference is had tothe equation 1 above. The time ruler 401 with display with the number offrames is graduated so that the number of frames is represented by fourdigits. The maximum number 9999 of frames is equivalent to approximately160 seconds.

[0320] As for the frame width setting on the picture in each case of thereal time display in seconds and the display with the number of framesf, reference is had to Tables 1 and 2 above.

[0321] The time ruler 401 includes an end time display field 405 and thecurrent time display field 406, in addition to the unit switching radiobutton 404.

[0322] In the end time display field 405, a time representing numericalfigure indicating the end time of the action being edited, that is theoperating time, is displayed. In the illustrated instance, “09:40” (=9seconds 40) is displayed. In the current time display field 306, a timerepresenting numerical figure for the current position is displayed. Inthe illustrated instance, “04:60” (=4 seconds 60) is displayed. Thesefields are editable text fields. When a numerical figure for anymeaningful time point is input, it becomes the end time such that thelast keyframe is moved or the current time is moved to such position.

[0323] On the time ruler 401, a ‘time width changing popup menu’, notshown, for changing the interval of the time ruler 401, may be invoked.

[0324] In the editing area, a ‘keyframe line 407’, an ‘end time line408’ and a ‘current time line 409’ are displayed as time display line.

[0325] The keyframe line 407, representing the time of each keyframe, aslater explained, is demonstrated as intersecting the respectivechannels. The user is able to perform the operation of editing the MIDIsound as the or she visually checks for synchronization with thekeyframe. Moreover, since the end time line 408 indicating the end timeof the action being edited is displayed as intersecting the respectivechannels, the user is able to visually comprehend the time range forediting. Moreover, the current time line 409, indicating the currenttime, is displayed as intersecting the respective channels. Basically,when any optional channel is clicked, the current time shifts to suchposition.

[0326] In the keyframe channel 402, keyframe positions, as acquired fromthe action editing window 200 along time axis as specified by the timeruler 401, are displayed. However, in distinction from the case of theaction editing window 200 (see FIG. 13), the keyframe channel cannot beopened nor closed within the sound detail window 400.

[0327] In the WAVE channel 403, the contents of the sound file of theWAVE form are represented as the waveform, as shown in FIG. 19. However,in distinction from the case of the score channel for MIDI style,described above, only the basic slicing operation is allowed on the WAVEchannel 403.

[0328] Additionally, a ‘replay button’ for commanding the replay of theedited sound may also be provided in the sound detail channel.

[0329]FIG. 22 schematically shows the structure of a sound detail window500 used for displaying and editing the LED operation file stated in theMIDI style. As shown, the sound detail window 500 is made up by a titlebar, a menu bar and an editing area in which to execute the editing ofthe LED operation file of the WAVE style.

[0330] In the menu bar, there are provided respective menus ‘file’,‘edit’, ‘set’ and ‘help’.

[0331] If the menu ‘file’ is selected, there appears a pull-down menucomprised of sub-menus ‘new LED operation’, ‘open LED operation’, ‘saveLED operation’, ‘save new LED operation’ and ‘close’ (see FIG. 23).

[0332] If the sub-menu ‘new LED operation’ is selected, a novel LEDoperation is produced. If already an unsaved MIDI sound is opened, thereappears a dialog which prompts the user to confirm whether or not theLED operation is to be saved.

[0333] If the sub-menu ‘open LED operation’ is selected, a pre-existingLED operation file is opened. If an unsaved MIDI sound has already beenopened, there appears a dialog prompting the user to confirm whether ornot its MIDI sound is to be saved (as above).

[0334] If the sub-menu ‘save LED operation’ is selected, the relevantLED operation file is overwrite-saved. In case of an unsaved file, afile setting dialog appears, as in the case of saving a new LEDoperation, as later explained, to prompt the user to input the filename.

[0335] If the sub-menu ‘save new LED operation’ is selected, a filesetting dialog appears to prompt the user to input a filename.

[0336] If the menu ‘close’ is selected, the LED operation detail windowis closed. If the LED operation file in the window is not as yet saved,there appears a dialog to prompt the user to confirm whether or not theunsaved file is to be saved.

[0337] If the menu ‘edit’ is selected, there appears a pulldown menucomprised of respective sub-menus of ‘revert’, ‘slice’, ‘copy’,‘overwrite paste’, ‘insert paste’ and ‘delete’ (see FIG. 17).

[0338] If the sub-menu ‘revert’ is selected, undo processing is carriedout beginning from the near-by operation.

[0339] If the sub-menu ‘slice’ is selected, the LED operation over aselected time span, if any, is cut. Actually, the so cut data istransiently stored in a clipboard. By this cutting, not the frame itselfbut the information concerning the contents of the frame is lost.

[0340] If the sub-menu ‘copy’ is selected, the LED operation over aselected time span, if any, is cut. Actually, the so copied data istransiently stored in a clipboard.

[0341] If the sub-menu ‘overwrite paste’ is selected, the contentsstored in the clipboard are pasted at the current time point.

[0342] If the sub-menu ‘insert paste’ is selected, the contents storedin the clipboard are inserted and pasted at the current time point.

[0343] If the sub-menu ‘delete’ is selected, the LED operation over aselected time span, if any, is deleted. It is not the frame itself butthe information on the contents of the frame that is lost.

[0344] In the menu ‘help’, there are included sub-menus, such as topic,support web or the version information.

[0345] Reverting to FIG. 22, the editing area of the LED detail window500 is explained. The editing area of the LED detail window for WAVEsound is a two-dimensional time-line table, formed by the time axis inthe transverse direction and by the channel in the vertical direction.The time line table is made up by a time ruler 501, a keyframe channel502 and a WAVE channel 503.

[0346] The time ruler 501 exploits a unit switching radio button 504 toswitch between real time display and the display with the number offrames. In the instance shown in FIG. 22, real-time display is selected.The real-time display is in second: millisecond, each being of twodigits. As for the relationship between the real time in the real-timedisplay and the display with the number of frames, reference is had tothe equation 1 above. The time ruler 501 with display with the number offrames is graduated so that the number of frames is represented by fourdigits. The maximum number 9999 of frames is equivalent to approximately160 seconds.

[0347] As for the frame width setting on the picture in each case of thereal time display in seconds and the display with the number of framesf, reference is had to Tables 1 and 2 above.

[0348] The time ruler 501 includes an end time display field 505 and acurrent time display field 506, in addition to the unit switching radiobutton 504. In the end time display field 505, a time representingnumerical figure indicating the end time of the action being edited,that is the operating time, is displayed. In the illustrated instance,“09:40” (=9 seconds 40) is displayed. In the current time display field506, a time representing numerical figure for the current position isdisplayed. In the illustrated instance, “04:60” (=4 seconds 60) isdisplayed. These fields are editable text fields. When a numericalfigure for any meaningful time point is input, it becomes the end timesuch that the last keyframe is moved or the current time is moved tosuch position.

[0349] On the time ruler 501, a ‘time width changing popup menu’, notshown, for changing the interval of the time ruler 501, may be invoked.

[0350] In the editing area, a ‘keyframe line 507’, an ‘end time line508’ and a ‘current time line 509’ are displayed as time display lines.The keyframe line 507, representing the time of each keyframe, as laterexplained, is demonstrated as intersecting the respective channels. Theuser is able to perform the operation of editing the LED operation asthe or she visually checks for synchronization with the keyframe.Moreover, since the end time line 508 indicating the end time of theaction being edited is displayed as intersecting the respectivechannels, the user is able to visually comprehend the time range forediting. Moreover, the current time line 509, indicating the currenttime, is displayed as intersecting the respective channels. Basically,when any optional channel is clicked, the current time shifts to suchposition.

[0351] In the keyframe channel 502, keyframe positions, as acquired fromthe action editing window 200 along time axis as specified by the timeruler 501, are displayed. However, in distinction from the case of theaction editing window 200 (see FIG. 13), the keyframe channel cannot beopened nor closed within the sound detail window 500.

[0352] The score channel is an area for editing the LED operation,stated in the MIDI style, by GUI operations, and is constituted by alist of mounting sites of LEDs on the main body unit of the mobile robot1, and by a basic grid in the time axis direction. In the presentembodiment, LEDs are arranged on the forehead, a right eye α, a left eyeα, aright eye β, a left eye β, a right eye γ, a left eye γ, a tail α anda tail β.

[0353] On the score channel, there are formed scores of respective sitelist by displaying the illuminated state of the LEDs of the respectivesites on the time axis. One mesh is termed a ‘cell’. The cells of thepositions of lighting of the LEDs on the time axis are colored dependingon the color and the lighting intensity. In distinction from the scorechannel for editing the MIDI sound, as described above with reference toFIG. 15, the LEDs of the respective sites can be turned on/offindependently of one another.

[0354] On the left lateral side of the score channel, there is displayedan LED site visual 510 which graphically represents the respective sitesof the LEDs that can be modified.

[0355] Below the LED site visual 510, there are arrayed velocity marksindicating rise, an uppermost position maintained, and descent. Thesemarks own reciprocally exclusive selection states such that one of themis selected at all times. The selection item is changed by mouse click.

[0356] The authoring system of the present embodiment includes a previewwindow 600 for visually confirming the contents of the action edited onthe action editing window 200.

[0357]FIG. 25 schematically shows the structure of the preview window600. As shown therein, the preview window 600 is made up of a ‘3D view601’, a ‘set of 3D display switching buttons’, a ‘current time field’606 and a ‘set of replay buttons’.

[0358] On the 3D preview 601 is perpetually displayed an image of thethree-dimensional mobile robot 1 generated by computer graphicsprocessing. The line of sight direction can be moved to change the‘look’ of the view by dragging on this view. Although not shown, the 3Cmodel can also be designed to be pre-viewed simultaneously from two ormore viewing points. The view movement is operatively linked to theuser's inputting operations on the set of 3D display switching buttons.There is also provided the function of checking for collision ofrespective sites or the driving speeds of the respective joints by this3D model, as an aid in creating and processing the 3D display of themotion. The center of gravity of the 3D model can be set on each keyframe to approximate the apparent movement of the 3D model to closer tothe movement of an actual machine.

[0359] On the right side of the 3D preview 601 is arranged an LEDoperation preview area for demonstrating the LED operations. In thispreview area, the manner of the turning on/off of the LEDs of the mobilerobot 1 is displayed in synchronism with the movements of the mobilerobot 1 on the 3D preview 601.

[0360] The set of the set of 3D display switching buttons includes a‘rotational button 602’, ‘zoom-in zoom-out button 603’, a ‘pan button’604 and a ‘home position button 605’. The user may click these buttonsto change the line of sight direction within the 3D preview 601.

[0361] For example, if the rotational button is clicked, the rotatingmode is set. If then the 3D preview 601 is dragged, the 3D mobile robot1 in the 3D preview 601 is rotated.

[0362] If then the ‘zoom-in zoom-out button 603’ is clicked, the 3Dpreview 601 is in the zoom mode. If then the 3D preview 601 is draggedin the vertical direction, the 3D mobile robot 1 in the 3D preview 601is zoomed in/out.

[0363] If then the ‘pan button 604’ is clicked, the 3D preview 601 is inthe pan mode. If the 3D view 601 then is dragged in the verticaldirection and in the left and right direction, the mobile robot 1 in the3D preview 601 is panned, that is moved at a high speed.

[0364] If the home position button 605 is clicked, the three-dimensionaldisplay of the mobile robot 1 reverts to a state of view of default,that is to a state of the robot as seen from the line of sightdirection.

[0365] In the current time field 606, the current time of the contentsas displayed on the 3D preview 601 is displayed. In the instance shown,the current time “04:60” is displayed. If any numerical figuremeaningful as time is input to ths field, the display on the 3D preview601 is changed to a frame of the relevant time. The current timeposition is also relatively visually displayed.

[0366] The set of replay buttons include a ‘frame rewind button 607’, a‘previous key frame button 608’, a ‘play/stop button 609’, a ‘stepwiseframe feed button 610’, a ‘frame feed button 611’ and a ‘loop replaybutton 612’.

[0367] If the ‘frame rewind button 607’ is clicked, the display of the3D preview 601 reverts to the first frame. If the ‘previous key framebutton 608’ is clicked, the replay of the display on the 3D preview 601skips from the current position to the directly previous keyframe. Ifthe ‘play/stop button 609’ is clicked, the replay of the display of the3D preview 601 is started or stopped (the play/stop button 609 is haltedor in operation during play or stop, respectively). The ‘stepwise framefeed button 610’ is valid only during reproduction of the display of the3D preview 601 and, if the button is clicked, the display isintermittently fed on the frame basis. When the ‘frame feed button 611’is clicked, the display of the 3D preview 601 skips to the last frame.When the ‘loop replay button 612’ is clicked, the 3D preview 601 isdisplayed in a loop fashion.

[0368] The authoring system of the instant embodiment includes a posewindow 700 for editing a three-dimensional pose of the mobile robot 1 byGUI operations consisting essentially in dragging.

[0369] The pose edited on the pose window 700 can be used e.g., as akeyframe making up the motion. For example, the pose window 700 can bebooted by double-clicking the desired keyframe on the keyframe channel.

[0370]FIG. 26 schematically shows the structure of the pose window 700.On this pose window 700, the rotational angles of the respective jointactuators forming the mobile robot 1 can be directly commanded by GUIoperations to specify the desired pose extremely readily. The posewindow 700 is made up of an entity specifying area 701, a listspecifying area 702, a setting value area 703, a 3D display area 704, aset of 3D display switching buttons 705 and a display switching popup706.

[0371] In the entity specifying area 701, a developed plan view of themobile robot 1 is displayed to permit the user to select the editablesites. The so selected sites correspond to the items of list commandsand are displayed with emphasis or ‘blinked’ on the 3D display to switchthe contents of the setting value areas.

[0372] In the list specifying area 702, the editable sites of the mobilerobot 1 and the setting values thereof are displayed as a list. If theuser selects a specified site from this list, the relevant site in theentity specifying area 701 is displayed with emphasis. In the 3D displayarea 704, emphasized display or blinking occurs to switch the contentsof the setting value area 703.

[0373] In the setting value area 703, the names of the setting sites,setting values, minimum settable values and maximum settable values ofthe editable sites are displayed in a list. If the user selects aspecified site, its contents are switched. The setting values can bedirectly set on key input in an inputtable field. The angular expressionis by an arc, with the setting value being varied on dragging a line forselection.

[0374] In the 3D display area 704, the full-body image of the mobilerobot 1 generated by 3D graphics is delineated along with theterrestrial ground. If the user clicks to select the relevant site fromthe 3D display, the site can be selected with emphasis. The settingvalue can be directly switched on dragging the selected site.

[0375] The contents displayed in the 3D display area 704 are operativelylinked with the 3D display switching button, such that, on dragging onthe view of the 3D display area 704, the ‘look’ of the view, that is theline of sight direction, can be changed.

[0376] The set of 3D display switching buttons includes a ‘rotationalbutton’, a ‘zoom-in zoom-out button’, a ‘pan button’ and a ‘homeposition button’. The user may click these buttons to change the line ofsight direction within the 3D display area 704.

[0377] For example, if the rotational button is clicked, the rotatingmode is set. If then the 3D display area 704 is dragged, the mobilerobot 1 in the 3D display area 704 is rotated. If then the zoom-inzoom-out button is clicked, the mode is the zoom mode. If then the 3Ddisplay area 704 is dragged in the vertical direction, the mobile robot1 is zoomed in and out in the 3D display area 704. If then the panbutton is clicked, the mode is the pan mode. If the mobile robot 1 thenis dragged in the vertical direction and in the left and right directionin the 3D display area 704, the mobile robot 1 in the 3D display area704 is panned, that is moved at a high speed. If the home positionbutton is clicked, the three-dimensional display of the mobile robot 1reverts to a state of view of default, that is to a state of the robotas seen from the line of sight direction.

[0378] On button clicking on the display switching popup 706, the apopup menu, not shown, comprised of a front side/back side/right lateralside/left lateral side/upper side/bottom side/3D is displayed forswitching to a view from the direction as selected by menu selection.

[0379] On the pose window 700, there are provided an OK button and acancel button. If the OK button is clicked, all editing items in thewindow are validated to close the window and, if conversely the cancelbutton is clicked, all editing items are invalidated to close thewindow, in a well-known manner.

[0380] Meanwhile, the pose window 700 can be used not only for editingthe pose of the mobile robot 1 on the authoring system, but also forreading in the orientation as taught to the mobile robot 1 on an actualdevice, that is joint values of the respective joint actuators, into thesystem for pre-viewing.

[0381]FIG. 27 shows a block diagram for illustrating the processingsequence for previewing the contents as directly taught on the actualdevice on the pose window 700. The operations of the flowchart arehereinafter explained.

[0382] First, an operator of the mobile robot 1 performs directteaching, such as by manually holding the trunk or legs on the actualmachine to cause the actual machine to strike a pose (step S1).

[0383] The setting values in the editable sites, such as jointactuators, obtained as a result of the direct teaching, are then read-inand transiently saved (step S2). The setting values are then transferredto the authoring system (step S3).

[0384] There is no particular limitation to the method of data transferto the authoring system. For example, near radio data communication,such as Bluetooth, may be used, or data transfer may be made betweendifferent devices over a recording medium, such as memory stick.

[0385] The authoring system reads-in the setting values of the editablesites (step S4) to open the pose window 700 to update the displaycontents or picture contents in the entity specifying area 701, listspecifying area 702, setting value area 703 3D and in the display area704 in accordance with the read-in setting values (step SS).

[0386] The authoring system of the present embodiment includes a motionpreviewer 800 for previewing the motion as edited by the motion channelas explained previously or the motion having the respective poses editedby the pose window as keyframes.

[0387]FIG. 28 schematically shows the structure of the motion previewer800. This motion previewer 800 can be opened on double-clicking themotion file displayed in a tree style on the project window.

[0388] On the motion previewer 800, the motion can be previewed, whileit can also be copied for pasting on the action editing window 200.

[0389] The motion previewer 800 is adapted for displaying a thumbnail ofone or more keyframes 801 making up the motion, that is the thumbnail ofthe poses. The keyframes 801 are arrayed e.g., in the chronologicalorder in which the motion is reproduced.

[0390] The keyframes in the motion previewer 800 may be dragged into theaction editing window 200. If there are too many keyframes making up themotion in the preview, horizontal scrolling is used.

[0391] [Supplement]

[0392] Although the present invention has been elucidated with referenceto specified embodiments thereof, the present invention can, of course,be corrected or substituted by those skilled in the art withoutdeparting from the scope of the invention.

[0393] Although the present embodiment is directed to an authoringsystem for a pet type robot walking on four legs, like a dog, this isnot intended to limit the present invention because the presentinvention can similarly be applied to a two-legged robot such as ahumanoid robot or to a mobile robot other than a legged robot.

[0394] Moreover, the ‘multi-joint’ as defined in the claim is notlimited to a physical apparatus typified in particular by a leggedrobot. For example, the authoring system of the present invention may,of course, be used for creating and editing the operating sequence foran animation employing a character generated by computer graphics.

[0395] The present invention has been elucidated only by way ofillustration and hence is not to be construed in a limiting fashion. Thepurport of the present invention is-to be verified in the light of thedescription in the claims.

INDUSTRIAL APPLICABILITY

[0396] According to the present invention, described above, there may beprovided an excellent authoring system and method capable of supportingthe creation and editing of the sequence of commands and data stating apreset pattern of operations for a robot.

[0397] According to the present invention, there may be provided anexcellent authoring system and method capable of supporting the creationand editing of the movement patterns can be supported using a set ofcomponents prescribing the operating states of the robot.

[0398] Moreover, according to the present invention, there may beprovided an excellent authoring system and method whereby the componentparts of the robot can be arrayed on the computer display to support thecreation and editing of the movement pattern.

[0399] In the authoring system and method according to the presentinvention, chronological data forming the action of the mobile robot,such as motion data, sound data or LED operation data, are arrayed alongthe time axis for display on a time table of a two-dimensional time linestyle. Consequently, such a working environment may be provided in whichcreation and editing may be made as the user visually checks forsynchronization between respective chronological data to warrant aworking environment of efficient and intuitively comprehensible actionedition.

[0400] In the authoring system and method according to the presentinvention, such a tool may be provided in which a multi-joint structure,including a robot first of all, may be handled as a new entertainment.According to the present invention, the behavior of the multi-jointstructure, including a robot in particular, can be programmed toformulate contents extremely readily even in the absence of the profoundknowledge about computer programming. For example, the user owns a toolfor expressing a multi-joint structure to expand the world offered bythe robot.

[0401] In the authoring system and method according to the presentinvention, a user is able to execute programming pertaining to thebehavioral sequence of a multi-joint structure through GUI operations.Moreover, the programing operations on the GUI picture may be furthersimplified and improved in efficiency by providing abundant library.

[0402] The operating sequence, that is behavior, of a robot, may beconstructed by unifying respective contents, that is motion, sound andthe LD operations. According to the present invention, such a workingenvironment may be provided which warrants facilitated synchronizationbetween the respective contents by exploiting a time line in an editingpicture of these respective contents. On the GUI picture according tothe present invention, the contents can not only be processed asindividual data but also treated as actions comprised of combinations ofdifferent contents.

1. An authoring system for supporting the creation and editing of anaction of a multi-joint structure comprised of a combination of aplurality of chronological data, comprising: a user inputting unit forinputting commands or data from a user; a user presenting unit forpresenting an editing area in which chronological data making up theaction are arrayed chronologically along the time axis; and achronological data editing unit provided for each chronological datamaking up the action, said chronological data editing unit creating orediting relevant chronological data based on a user input through saiduser presenting unit.
 2. The authoring system according to claim 1wherein one of the chronological data making up the action is motiondata stating the chronological movements of respective joints of themulti-joint structure.
 3. The authoring system according to claim 1wherein one of the chronological data making up the action is motiondata which prescribes chronological movements of respective joints ofthe multi-joint structure and which, by arraying two or more keyframeseach representing the multi-joint structure striking a preset pose onthe time axis, smoothly joins the respective keyframes.
 4. The authoringsystem according to claim 1 wherein one of the chronological data makingup the action is sound data output as sound in timed relation toreproduction of said action.
 5. The authoring system according to claim4 wherein the sound data is stated in the MIDI (Musical InstrumentalDigital Interface) form or WAVE form.
 6. The authoring system accordingto claim 1 wherein one of the chronological data making up the action isindicator display data stating the turning on/off of display indicatorsdisplayed in timed relation to reproduction of said action.
 7. Theauthoring system according to claim 1 wherein indicator display data isstated in the MIDI (Musical Instrumental Digital Interface) form.
 8. Theauthoring system according to claim 1 wherein the editing area presentedby said user presenting unit is comprised of chronological data basedchronological data display channels, said data extending along thehorizontally arrayed time axis, said channels being arrayed in thevertical direction.
 9. The authoring system according to claim 8 whereinthe editing area presented by said user presenting unit includes a timeruler made up by graduations representing the time axis in real-time.10. The authoring system according to claim 8 wherein the editing areapresented by said user presenting unit includes one or more time displaylines extending in the vertical direction for indicating the relevanttime prescribed by a time ruler.
 11. The authoring system according toclaim 8 wherein the editing area presented by said user presenting unitincludes a vertically extending current time display line for indicatingthe current time on the time axis prescribed by the time ruler; saidcurrent time display line being shifted on the time ruler to a positioncorresponding to user actuation.
 12. The authoring system according toclaim 8 wherein the editing area presented by said user presenting unitincludes a display window for previewing movements of the multi-jointstructure at the current time.
 13. The authoring system according toclaim 8 wherein one of the chronological data making up the action ismotion data which prescribes the chronological movements of respectivejoints of the multi-joint structure and which arrays two or morekeyframes, each representing the multi-joint structure striking a presetpose, to smoothly join the respective keyframes; said editing areapresented by said user presenting unit including a keyframe channelrepresenting each keyframe or its thumbnail along the time axisprescribed by said time ruler.
 14. The authoring system according toclaim 13 wherein the keyframe time is changed so as to follow up withdragging of said keyframe or its thumbnail within the keyframe channel.15. The authoring system according to claim 13 wherein a pose editingpicture for editing a relevant pose is started responsive to a selectingoperation by a user of a keyframe or its thumbnail within the keyframechannel.
 16. The authoring system according to claim 8 wherein one ofthe chronological data making up the action is motion data stating thechronological movements of respective joints of the multi-jointstructure; the editing area presented by said user presenting unitincludes a motion channel for editing and displaying the contents of amotion along the time axis prescribed by the time ruler.
 17. Theauthoring system according to claim 16 wherein said motion channel iscomprised of respective timing charts representing chronologicalmovements of respective joints of the multi-joint structure, said timingcharts being arrayed in the vertical direction.
 18. The authoring systemaccording to claim 17 wherein the movements of joints at a relevanttiming are changed so as to follow up with dragging on the timing chartin said motion channel.
 19. The authoring system according to claim 8wherein one of the chronological data making up the action is sound dataissued as sound in timed relation to reproduction of an action; theediting area presented by said user presenting unit including a soundchannel for indicating the contents of the sound along the time axisprescribed by the time ruler.
 20. The authoring system according toclaim 8 wherein one of the chronological data making up the action isindicator display data stating the operation of the turning on/off ofdisplay indicators, as output in timed relation to the reproduction ofan action; the editing area presented by said user presenting unitincluding a display indicator channel for indicating the contents of theindicator data along the time axis prescribed by the time ruler.
 21. Theauthoring system according to claim 1 wherein one of the chronologicaldata making up the action is sound data output as sound in timedrelation to the reproduction of an action; said user presenting unitfurther indicating a sound editing area for indicating and editing thesound contents along the time axis prescribed by said time ruler. 22.The authoring system according to claim 21 wherein said sound editingarea includes a score channel constituted by a basic grid along the timeaxis direction and a piano keyboard; a score being formed by the soundduration as the reference of the time axis and the pitch of the pianokeys; the sound being edited by arraying the color corresponding tosound notes in a cell corresponding to the desired time point and thesound scale.
 23. The authoring system according to claim 21 wherein saidsound editing area includes a velocity channel for representing theintensity of each sound along the time axis.
 24. The authoring systemaccording to claim 1 wherein one of the chronological data making up anaction is indicator data stating the turning on/off of an indicator,output in timed relation to the reproduction of an action; said userpresenting unit further indicating an indicator editing area fordisplaying and editing the contents of indicator data along the timeaxis prescribed by the time ruler.
 25. The authoring system according toclaim 24 wherein said indicator editing area includes a score channelconstituted by a list of sites of arraying of display indicators and bythe basic grid along the time axis direction; a score being edited fromone site list to another by indicating the turned-on state of theindicator in each site on the time axis on a score channel.
 26. Theauthoring system according to claim 1 wherein said user presenting unitfurther displays a preview window for visually checking an action of themulti-joint structure generated based on each chronological data asedited by the chronological data editing unit.
 27. The authoring systemaccording to claim 26 wherein one of the chronological data making up anaction is motion data stating the chronological movements of therespective joints of the multi-joint structure; said preview windowincluding a 3D view area for three-dimensional display of movements ofthe multi-joint structure generated based on motion data.
 28. Theauthoring system according to claim 26 wherein one of the chronologicaldata making up an action is indicator data stating the turning on/off ofan indicator, output in timed relation to the reproduction of an action;said preview window including an indicator operation preview area fordisplaying the operation of the indicators corresponding to theindicator display data in synchronism with the preview of otherchronological data.
 29. The authoring system according to claim 1wherein said user presenting unit further displays a pose window forediting the pose of the multi-joint structure by the GUI operation. 30.The authoring system according to claim 29 wherein said pose windowincludes an entity specifying area for displaying the multi-jointstructure in a developed plan view for accepting user selection ofeditable sites.
 31. The authoring system according to claim 29 whereinsaid pose window includes a list specifying area for displaying a listof editable sites of the multi-joint structure and setting valuesthereof.
 32. The authoring system according to claim 29 wherein saidpose window includes a setting value area for displaying a list of thenames of setting sites, setting values, settable maximum values andsettable minimum values of editable sites of the multi-joint structure.33. The authoring system according to claim 29 wherein said pose windowincludes a 3D display area for 3D displaying a full image of themulti-joint structure generated by 3D graphics and for accepting userselection on said 3D display.
 34. The authoring system according toclaim 29 further comprising: data inputting means for inputtingchronological data making up an action of the multi-joint structure fromoutside; said pose window displaying the pose generated based on datainput from said data inputting means.
 35. The authoring system accordingto claim 1 wherein one of the chronological data making up an action ismotion data which prescribes chronological movements of respectivejoints of the multi-joint structure and which arrays two or morekeyframes each representing the multi-joint structure striking a presetpose to smoothly join the respective keyframes; said user presentingunit further displaying a motion preview window arraying one or more keyframes making up the motion or thumbnails thereof in accordance with thechronological sequence used in motion reproduction.
 36. An authoringmethod for supporting the creation and editing of an action of amulti-joint structure comprised of a combination of a plurality ofchronological data, comprising: a user presenting step of presenting anediting area in which the chronological data making up the action arearrayed chronologically along the time axis; and a chronological dataediting step for creating or editing relevant chronological data basedon a user input through the editing area, provided for eachchronological data making up the action, via said user presenting step.37. The authoring method according to claim 36 wherein one of thechronological data making up the action is motion data stating thechronological movements of respective joints of the multi-jointstructure.
 38. The authoring method according to claim 36 wherein one ofthe chronological data making up the action is motion data whichprescribes chronological movements of respective joints of themulti-joint structure and which, by arraying two or more keyframes eachrepresenting the multi-joint structure striking a preset pose on thetime axis, smoothly joins the respective keyframes.
 39. The authoringmethod according to claim 36 wherein one of the chronological datamaking up the action is sound data output as sound in timed relation toreproduction of said action.
 40. The authoring method according to claim39 wherein the sound data is stated in the MIDI (Musical InstrumentalDigital Interface) form or WAVE form.
 41. The authoring method accordingto claim 36 wherein one of the chronological data making up the actionis indicator display data stating the turning on/off of displayindicators, said display data being displayed in timed relation toreproduction of said action.
 42. The authoring method according to claim36 wherein indicator display data is stated in the MIDI (MusicalInstrumental Digital Interface) form.
 43. The authoring method accordingto claim 36 wherein the editing area presented in said user presentingstep is comprised of chronological data based chronological data displaychannels, said data extending along the horizontally arrayed time axis,said channels being arrayed in the vertical direction.
 44. The authoringmethod according to claim 43 wherein the editing area presented in saiduser presenting step includes a time ruler made up by graduationsrepresenting the time axis in real-time.
 45. The authoring methodaccording to claim 43 wherein the editing area presented by said userpresenting step includes one or more time display lines extending in thevertical direction for indicating the relevant time prescribed by a timeruler.
 46. The authoring method according to claim 43 wherein theediting area presented by said user presenting step includes avertically extending current time display line for indicating thecurrent time on the time axis prescribed by the time ruler; there beingincluded a further step of shifting said current time display line onthe time ruler to a position corresponding to user actuation.
 47. Theauthoring method according to claim 43 further comprising a step ofpresenting a display window for previewing movements of the multi-jointstructure at the current time.
 48. The authoring method according toclaim 43 wherein one of the chronological data making up the action ismotion data which prescribes the chronological movements of respectivejoints of the multi-joint structure and which arrays two or morekeyframes each representing the multi-joint structure striking a presetpose to smoothly join the respective keyframes; said user presentingstep including presenting a keyframe channel representing each keyframeor its thumbnail along the time axis prescribed by said time ruler. 49.The authoring method according to claim 48 wherein the keyframe time ischanged so as to follow up with dragging of said keyframe or itsthumbnail within the keyframe channel.
 50. The authoring methodaccording to claim 48 wherein a pose editing picture for editing arelevant pose is started responsive to a user selecting operation of akeyframe or its thumbnail within the keyframe channel.
 51. The authoringmethod according to claim 43 wherein one of the chronological datamaking up the action is motion data stating the chronological movementsof respective joints of the multi-joint structure; said user presentingstep including a motion channel for editing and displaying the contentsof a motion along the time axis prescribed by the time ruler.
 52. Theauthoring method according to claim 51 wherein said user presenting steprepresents a motion channel by arraying respective timing chartsrepresenting chronological movements of respective joints of themulti-joint structure in the vertical direction.
 53. The authoringmethod according to claim 52 further comprising a step of changing themovements of joints at a relevant time so as to follow up with draggingon the timing chart in said motion channel.
 54. The authoring methodaccording to claim 43 wherein one of the chronological data making upthe action is sound data issued as sound in timed relation toreproduction of an action; said user presenting step presenting anediting area including a sound channel for indicating the contents ofthe sound along the time axis prescribed by the time ruler.
 55. Theauthoring method according to claim 43 wherein one of the chronologicaldata making up the action is indicator data stating the operation of theturning on/off of the indicator, as output in timed relation to thereproduction of an action; said user presenting step presenting anediting area including a display indicator channel for indicating thecontents of the indicator display data along the time axis prescribed bythe time ruler.
 56. The authoring method according to claim 36 whereinone of the chronological data making up the action is sound data outputas sound in timed relation to the reproduction of an action; there beingfurther provided a step of indicating a sound editing area forindicating and editing the sound contents along the time axis prescribedby said time ruler.
 57. The authoring method according to claim 56wherein said sound editing area includes a score channel constituted bya basic grid along the time axis direction and a piano keyboard; a scorebeing formed by the sound duration as the reference of the time axis andthe pitch of the piano keys; the sound being edited by arraying thecolor corresponding to sound notes in a cell corresponding to thedesired time point and the sound scale on said score channel.
 58. Theauthoring method according to claim 56 wherein said sound editing areaincludes a velocity channel for representing the intensity of each soundalong the time axis.
 59. The authoring method according to claim 36wherein one of the chronological data making up an action is indicatordisplay data stating the turning on/off of a display indicator, outputin timed relation to the reproduction of an action; there being furtherprovided a step of indicating an indicator editing area for displayingand editing the contents of indicator data along the time axisprescribed by the time ruler.
 60. The authoring method according toclaim 59 wherein said indicator editing area includes a score channelconstituted by a list of arraying sites of the display indicators and bythe basic grid along the time axis direction; a score being edited fromone site list to another by indicating the turned-on state of thedisplay indicators in respective sites on the time axis on a scorechannel.
 61. The authoring method according to claim 36 furthercomprising a step of displaying a preview window for visually checkingan action of the multi-joint structure generated based on eachchronological data as edited in the chronological data editing step. 62.The authoring method according to claim 61 wherein one of thechronological data making up an action is motion data stating thechronological movements of respective joints of the multi-jointstructure; said preview window indicating step three-dimensionallydisplaying movements of the multi-joint structure, generated based onmotion data, in a 3D view area.
 63. The authoring method according toclaim 61 wherein one of the chronological data making up an action isindicator data stating the turning on/off of display indicators, outputin timed relation to the reproduction of an action; said preview windowindicating step displaying the operation of the display indicatorscorresponding to the indicator display data in synchronism with thepreview of other chronological data.
 64. The authoring method accordingto claim 36 further comprising a step of displaying a pose window forediting the pose of the multi-joint structure by the GUI operation. 65.The authoring method according to claim 64 wherein said pose windowincludes an entity specifying area for displaying the multi-jointstructure in a developed plan view for accepting user selection ofeditable sites.
 66. The authoring method according to claim 64 whereinsaid pose window includes a list specifying area for displaying a listof editable sites of the multi-joint structure and setting valuesthereof.
 67. The authoring method according to claim 64 wherein saidpose window includes a setting value area for displaying a list of thenames of setting sites, setting values, settable maximum values andsettable minimum values of editable sites of the multi-joint structure.68. The authoring method according to claim 64 wherein said pose windowincludes a 3D display area for 3D displaying a full image of themulti-joint structure generated by 3D graphics and for accepting userselection of said editable sites on said 3D display.
 69. The authoringmethod according to claim 64 further comprising a data inputting step ofinputting chronological data making up an action of the multi-jointstructure from outside; said pose window displaying the pose generatedbased on data input from said data inputting means.
 70. The authoringmethod according to claim 36 wherein one of the chronological datamaking up an action is motion data which prescribes chronologicalmovements of respective joints of the multi-joint structure and whicharrays two or more keyframes each representing the multi-joint structurestriking a preset pose to smoothly join the respective keyframes; therebeing further provided a step of displaying a motion preview windowarraying one or more key frames making up the motion or thumbnailsthereof in accordance with the chronological sequence used in motionreproduction.
 71. A recording medium having physically stored thereon,in a computer-readable form, a computer software stating the processingfor supporting the creation and editing of an action of a multi-jointstructure, comprised of the combination of plural chronological data, ona computer system, said computer software comprising: a user presentingstep of presenting an editing area in which chronological data making upthe action are arrayed chronologically along the time axis; and achronological data editing step for creating or editing relevantchronological data based on a user input through an editing area by saiduser presenting step, said editing area being provided for eachchronological data making up the action.